Summary
In this paper, the authors aim to enhance the adaptive aggregation in a customized way for personalized federated learning. They proposed an algorithm-unrolling-based method, i.e., Learn2pFed, to adaptively choose the part of parameters and the degree in aggregation. To validate Learn2pFed, they conduct extensive tasks, i.e., regression, forecasting, and image classification.
Strengths
Strengths:
I enjoy the insights of this work for (1) clear presentation, (2) methodology design, and (3) the extensive experiments.
S1: The authors present this paper with clear illustration and systematic logic outline for optimizing global model and client models.
S2: The mechanism of enhancing both the parts and the degree of model parameters is reasonable for enhancing performance. Besides, the proposed Learn2pFed method learns hyper-parameters via algorithm unrolling, which is more flexible.
S3: It interests us for conducting extensive experiments in different tasks, i.e., regression, forecasting, and image classification.
Weaknesses
I feel it uncertain and weak from the aspects of: (1) the theoretic analysis, (2) the privacy leakage, (3) the novelty of customized aggregation, and (4) the reproducibility.
W1: The convergence bound of Learn2pFed is not provided, and complexity analysis related to stability is deficient, which is necessary to clarify the concerns of additional computation and memory burden, since in Fig.5 appendix relies on more than 50 iterations.
W2: The privacy enhancement is limited. Iterative optimization, e.g., ADMM, relies on the gradient exchange among clients and servers. And the authors claim that a linear combination of multiple local variables can achieve the goal of privacy-preserving. However, no empirical and theoretic analysis is present.
W3: The related work is insufficiently studied in empirical results. Personalized and adaptive aggregation have been studied in (1) choosing model parameters parts[3,4], and (2) adaptive the degree of model aggregation [1,2,5]. It is necessary to compare the difference among these methods for highlighting the contribution of considering both.
[1] Li Z, Lin T, Shang X, et al. Revisiting weighted aggregation in federated learning with neural networks[J]. arXiv preprint arXiv:2302.10911, 2023.
[2] Zhang J, Hua Y, Wang H, et al. FedALA: Adaptive local aggregation for personalized federated learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2023, 37(9): 11237-11244.
[3] Lu W, Hu X, Wang J, et al. FedCLIP: Fast Generalization and Personalization for CLIP in Federated Learning[J]. arXiv preprint arXiv:2302.13485, 2023.
[4] Isik B, Pase F, Gunduz D, et al. Sparse random networks for communication-efficient federated learning[J]. arXiv preprint arXiv:2209.15328, 2022.
[5] Liao, Xinting, et al. "HyperFed: Hyperbolic Prototypes Exploration with Consistent Aggregation for Non-IID Data in Federated Learning." arXiv preprint arXiv:2307.14384 (2023).
W4: The implementation code is not open-source, which brings me three concerns, i.e., (1) the performance generalization of Learn2pFed in more complex datasets, (2) the additional computational burden for the newly proposed method, and (3) the capability of privacy in attack.
Questions
Q1: Since the existing analysis of convergence, computation and memory are all depended on the empirical studies with regard to small datasets, could the authors provide us with more theoretic analysis.
Q2: Could the authors provide us with related privacy defense analysis?
Q3: How can Learn2pFed become more scalable to large and complex federated setting, which is more practical in real-world applications?
Rating
3: reject, not good enough
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.