With the evolution of edge intelligence technologies, federated learning has become a prominent decentralized learning framework, facilitating joint model training among numerous devices without compromising data privacy. However, federated learning confronts several challenges in practical applications, including data heterogeneity and edge device instability. To tackle these issues, this article introduces pFedMLKD, an innovative personalized federated learning framework that incorporates hierarchical knowledge distillation and leverages historical global model insights to enhance local model training, thereby reducing the effects of data heterogeneity. This method enhances learning efficiency and improves the generalization of models within federated learning systems. We establish theoretical guarantees for pFedMLKD’s optimization under nonconvex settings through rigorous convergence analysis. Empirical validation encompasses three benchmark datasets (CIFAR-10, CIFAR-100, and EMNIST) under strict nonindependent and identically distributed data partitioning protocols. The results demonstrate that our algorithm outperforms current methods in federated learning scenarios.
Paper
Full text
pFedMLKD: A Novel Framework for Personalized Federated Learning via Multilevel Distillation
OpenAlex · Privacy-Preserving Technologies in Data · 2025
Abstract
With the evolution of edge intelligence technologies, federated learning has become a prominent decentralized learning framework, facilitating joint model training among numerous devices without compromising data privacy. However, federated learning confronts several challenges in practical applications, including data heterogeneity and edge device instability. To tackle these issues, this article introduces pFedMLKD, an innovative personalized federated learning framework that incorporates hierarchical knowledge distillation and leverages historical global model insights to enhance local model training, thereby reducing the effects of data heterogeneity. This method enhances learning efficiency and improves the generalization of models within federated learning systems. We establish theoretical guarantees for pFedMLKD’s optimization under nonconvex settings through rigorous convergence analysis. Empirical validation encompasses three benchmark datasets (CIFAR-10, CIFAR-100, and EMNIST) under strict nonindependent and identically distributed data partitioning protocols. The results demonstrate that our algorithm outperforms current methods in federated learning scenarios.