Addressing Algorithmic Disparity and Performance Inconsistency in Federated Learning

Federated learning (FL) has gain growing interests for its capability of\nlearning from distributed data sources collectively without the need of\naccessing the raw data samples across different sources. So far FL research has\nmostly focused on improving the performance, how the algorithmic disparity will\nbe impacted for the model learned from FL and the impact of algorithmic\ndisparity on the utility inconsistency are largely unexplored. In this paper,\nwe propose an FL framework to jointly consider performance consistency and\nalgorithmic fairness across different local clients (data sources). We derive\nour framework from a constrained multi-objective optimization perspective, in\nwhich we learn a model satisfying fairness constraints on all clients with\nconsistent performance. Specifically, we treat the algorithm prediction loss at\neach local client as an objective and maximize the worst-performing client with\nfairness constraints through optimizing a surrogate maximum function with all\nobjectives involved. A gradient-based procedure is employed to achieve the\nPareto optimality of this optimization problem. Theoretical analysis is\nprovided to prove that our method can converge to a Pareto solution that\nachieves the min-max performance with fairness constraints on all clients.\nComprehensive experiments on synthetic and real-world datasets demonstrate the\nsuperiority that our approach over baselines and its effectiveness in achieving\nboth fairness and consistency across all local clients.\n