FedDR -- Randomized Douglas-Rachford Splitting Algorithms for Nonconvex Federated Composite Optimization

We develop two new algorithms, called, FedDR and asyncFedDR, for solving a\nfundamental nonconvex composite optimization problem in federated learning. Our\nalgorithms rely on a novel combination between a nonconvex Douglas-Rachford\nsplitting method, randomized block-coordinate strategies, and asynchronous\nimplementation. They can also handle convex regularizers. Unlike recent methods\nin the literature, e.g., FedSplit and FedPD, our algorithms update only a\nsubset of users at each communication round, and possibly in an asynchronous\nmanner, making them more practical. These new algorithms can handle statistical\nand system heterogeneity, which are the two main challenges in federated\nlearning, while achieving the best known communication complexity. In fact, our\nnew algorithms match the communication complexity lower bound up to a constant\nfactor under standard assumptions. Our numerical experiments illustrate the\nadvantages of our methods over existing algorithms on synthetic and real\ndatasets.\n