Robbing the Fed: Directly Obtaining Private Data in Federated Learning with Modified Models

Federated learning has quickly gained popularity with its promises of\nincreased user privacy and efficiency. Previous works have shown that federated\ngradient updates contain information that can be used to approximately recover\nuser data in some situations. These previous attacks on user privacy have been\nlimited in scope and do not scale to gradient updates aggregated over even a\nhandful of data points, leaving some to conclude that data privacy is still\nintact for realistic training regimes. In this work, we introduce a new threat\nmodel based on minimal but malicious modifications of the shared model\narchitecture which enable the server to directly obtain a verbatim copy of user\ndata from gradient updates without solving difficult inverse problems. Even\nuser data aggregated over large batches -- where previous methods fail to\nextract meaningful content -- can be reconstructed by these minimally modified\nmodels.\n