Joint Superposition Coding and Training for Federated Learning over Multi-Width Neural Networks

This paper aims to integrate two synergetic technologies, federated learning\n(FL) and width-adjustable slimmable neural network (SNN) architectures. FL\npreserves data privacy by exchanging the locally trained models of mobile\ndevices. By adopting SNNs as local models, FL can flexibly cope with the\ntime-varying energy capacities of mobile devices. Combining FL and SNNs is\nhowever non-trivial, particularly under wireless connections with time-varying\nchannel conditions. Furthermore, existing multi-width SNN training algorithms\nare sensitive to the data distributions across devices, so are ill-suited to\nFL. Motivated by this, we propose a communication and energy-efficient\nSNN-based FL (named SlimFL) that jointly utilizes superposition coding (SC) for\nglobal model aggregation and superposition training (ST) for updating local\nmodels. By applying SC, SlimFL exchanges the superposition of multiple width\nconfigurations that are decoded as many as possible for a given communication\nthroughput. Leveraging ST, SlimFL aligns the forward propagation of different\nwidth configurations, while avoiding the inter-width interference during\nbackpropagation. We formally prove the convergence of SlimFL. The result\nreveals that SlimFL is not only communication-efficient but also can counteract\nnon-IID data distributions and poor channel conditions, which is also\ncorroborated by simulations.\n