The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation

We consider training models on private data that are distributed across user\ndevices. To ensure privacy, we add on-device noise and use secure aggregation\nso that only the noisy sum is revealed to the server. We present a\ncomprehensive end-to-end system, which appropriately discretizes the data and\nadds discrete Gaussian noise before performing secure aggregation. We provide a\nnovel privacy analysis for sums of discrete Gaussians and carefully analyze the\neffects of data quantization and modular summation arithmetic. Our theoretical\nguarantees highlight the complex tension between communication, privacy, and\naccuracy. Our extensive experimental results demonstrate that our solution is\nessentially able to match the accuracy to central differential privacy with\nless than 16 bits of precision per value.\n