NoPeek: Information leakage reduction to share activations in distributed deep learning

For distributed machine learning with sensitive data, we demonstrate how\nminimizing distance correlation between raw data and intermediary\nrepresentations reduces leakage of sensitive raw data patterns across client\ncommunications while maintaining model accuracy. Leakage (measured using\ndistance correlation between input and intermediate representations) is the\nrisk associated with the invertibility of raw data from intermediary\nrepresentations. This can prevent client entities that hold sensitive data from\nusing distributed deep learning services. We demonstrate that our method is\nresilient to such reconstruction attacks and is based on reduction of distance\ncorrelation between raw data and learned representations during training and\ninference with image datasets. We prevent such reconstruction of raw data while\nmaintaining information required to sustain good classification accuracies.\n