Personalized Federated Learning: A Unified Framework and Universal Optimization Techniques

We investigate the optimization aspects of personalized Federated Learning\n(FL). We propose general optimizers that can be applied to numerous existing\npersonalized FL objectives, specifically a tailored variant of Local SGD and\nvariants of accelerated coordinate descent/accelerated SVRCD. By examining a\ngeneral personalized objective capable of recovering many existing personalized\nFL objectives as special cases, we develop a comprehensive optimization theory\napplicable to a wide range of strongly convex personalized FL models in the\nliterature. We showcase the practicality and/or optimality of our methods in\nterms of communication and local computation. Remarkably, our general\noptimization solvers and theory can recover the best-known communication and\ncomputation guarantees for addressing specific personalized FL objectives.\nConsequently, our proposed methods can serve as universal optimizers, rendering\nthe design of task-specific optimizers unnecessary in many instances.\n