Federated Expectation Maximization with heterogeneity mitigation and variance reduction

The Expectation Maximization (EM) algorithm is the default algorithm for\ninference in latent variable models. As in any other field of machine learning,\napplications of latent variable models to very large datasets make the use of\nadvanced parallel and distributed architectures mandatory. This paper\nintroduces FedEM, which is the first extension of the EM algorithm to the\nfederated learning context. FedEM is a new communication efficient method,\nwhich handles partial participation of local devices, and is robust to\nheterogeneous distributions of the datasets. To alleviate the communication\nbottleneck, FedEM compresses appropriately defined complete data sufficient\nstatistics. We also develop and analyze an extension of FedEM to further\nincorporate a variance reduction scheme. In all cases, we derive finite-time\ncomplexity bounds for smooth non-convex problems. Numerical results are\npresented to support our theoretical findings, as well as an application to\nfederated missing values imputation for biodiversity monitoring.\n

Paper

Similar papers

© 2026 NYSGPT2525 LLC