Neither Private Nor Fair: Impact of Data Imbalance on Utility and Fairness in Differential Privacy

Deployment of deep learning in different fields and industries is growing day\nby day due to its performance, which relies on the availability of data and\ncompute. Data is often crowd-sourced and contains sensitive information about\nits contributors, which leaks into models that are trained on it. To achieve\nrigorous privacy guarantees, differentially private training mechanisms are\nused. However, it has recently been shown that differential privacy can\nexacerbate existing biases in the data and have disparate impacts on the\naccuracy of different subgroups of data. In this paper, we aim to study these\neffects within differentially private deep learning. Specifically, we aim to\nstudy how different levels of imbalance in the data affect the accuracy and the\nfairness of the decisions made by the model, given different levels of privacy.\nWe demonstrate that even small imbalances and loose privacy guarantees can\ncause disparate impacts.\n