Differentially Private Federated Bayesian Optimization with Distributed Exploration

Bayesian optimization (BO) has recently been extended to the federated\nlearning (FL) setting by the federated Thompson sampling (FTS) algorithm, which\nhas promising applications such as federated hyperparameter tuning. However,\nFTS is not equipped with a rigorous privacy guarantee which is an important\nconsideration in FL. Recent works have incorporated differential privacy (DP)\ninto the training of deep neural networks through a general framework for\nadding DP to iterative algorithms. Following this general DP framework, our\nwork here integrates DP into FTS to preserve user-level privacy. We also\nleverage the ability of this general DP framework to handle different parameter\nvectors, as well as the technique of local modeling for BO, to further improve\nthe utility of our algorithm through distributed exploration (DE). The\nresulting differentially private FTS with DE (DP-FTS-DE) algorithm is endowed\nwith theoretical guarantees for both the privacy and utility and is amenable to\ninteresting theoretical insights about the privacy-utility trade-off. We also\nuse real-world experiments to show that DP-FTS-DE achieves high utility\n(competitive performance) with a strong privacy guarantee (small privacy loss)\nand induces a trade-off between privacy and utility.\n