A-FedPD: Aligning Dual-Drift is All Federated Primal-Dual Learning Needs

As a popular paradigm for juggling data privacy and collaborative training, federated learning (FL) is flourishing to distributively process the large scale of heterogeneous datasets on edged clients. Due to bandwidth limitations and security considerations, it ingeniously splits the original problem into multiple subproblems to be solved in parallel, which empowers primal dual solutions to great application values in FL. In this paper, we review the recent development of classical federated primal dual methods and point out a serious common defect of such methods in non-convex scenarios, which we say is a"dual drift"caused by dual hysteresis of those longstanding inactive clients under partial participation training. To further address this problem, we propose a novel Aligned Federated Primal Dual (A-FedPD) method, which constructs virtual dual updates to align global consensus and local dual variables for those protracted unparticipated local clients. Meanwhile, we provide a comprehensive analysis of the optimization and generalization efficiency for the A-FedPD method on smooth non-convex objectives, which confirms its high efficiency and practicality. Extensive experiments are conducted on several classical FL setups to validate the effectiveness of our proposed method.

Paper

References (63)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer c2hH6/10 · confidence 4/52024-06-28

Summary

The authors propose an algorithm for distributed optimization of a sum of nonconvex smooth functions, with partial participation. They obtain a O(1/T) rate.

Strengths

The study of convergence of algorithms adapted to federated learning is an important topic.

Weaknesses

* There is no discussion of the O(1/T) rate in Theorem 1 with respect to existing results, for instance on Scaffold. I don't think the method improves on the SOTA. * In Federated Learning (FL), the bottleneck is communication. The complexity in number of communication rounds is only one part of it, sending full vectors is not reasonable in FL, compression must be used to reduce dramatically the number of communicated bits, which is the right criterion to study. * There is a lack of comparison to existing methods. Here the local steps are performed to find a stationary point of the augmented Lagrangian. This is equivalent to compute, inexactly, a proximity operator. This is exactly what the 5GCS algorithm does, proposed in Grudzien et al. "Can 5th generation local training methods support client sampling? yes!" AISTATS 2023. 5GCS has been extended to compression in Grudzien et al "Improving Accelerated Federated Learning with Compression and Importance Sampling", arXiv:2306.03240. Another technique is loopless: instead of an inner loop of local steps, communication is triggered randomly. This is the technique used in Scaffnew, an important algorithm since which demonstrates that local training yields acceleration of communication in FL: Mishchenko et al. "ProxSkip: Yes! Local gradient steps provably lead to communication acceleration! Finally!" ICML 2022. Scaffnew has been interpreted as a primal-dual algorithm in Condat and Richtárik, "RandProx: Primal-Dual Optimization Algorithms with Randomized Proximal Updates" ICLR 2023. Scaffnew has been extended to partial participation, and also compression, as TAMUNA in Condat et al. "TAMUNA: Doubly Accelerated Federated Learning with Local Training, Compression, and Partial Participation" arXiv:2302.09832. * In these algorithms, such as 5GCS and TAMUNA, the dual variables of the inactive clients are not modified. This is fine, because the aggregated model from the active clients does not contain any information about the inactive clients, so we cannot expect to use the new model to update these dual variables. So, I don't see what the challenge addressed in the paper is, because combining local training and partial participation can been done successfully, as these papers show. * The writing is not good. The first 3 pages contain only vague statements, without the problem being defined. The paper contains a lot of unusual or weird expressions, such as "imminent challenge" "brings great distress for training". What does the sentence "Ozfatura et al. [2021], Xu et al. [2021] propose to adopt the global update consistently control the local training to force a high consensus." mean? * I don't see what "dual drift" actually means. If the dual variables of inactive clients remain the same for a long time, this is not a drift. The only explanation of this notion is that there is "excessive differences between the primal and dual variables", which is unclear, one cannot compare a primal and dual variable. If you mean that the primal and dual variables become inconsistent with each other, this inconsistency should be made precise. So, the paper title should be modified. Also, it is not grammatically correct, this should be "Needs", not "Need". Typos * (5) first update: this should be $\theta_i$, not $\theta_i^t$ * Algorithm 1, line 4: this should be $P^t$, not $N^t$

Questions

* Theorem 1 "let rho > O(L^3) be positive with lower bound": what does it mean ? * rho proportional to $L^3$ is weird, the left hand side and right hand side of (8) are not homogeneous. Looking at the proof, Line 987: is is not clear at all what the condition on rho is. * line 3: "randomly select active clients set" : do you mean choose a subset of size P uniformly at random? * Why does the number P of active clients not appear in Theorem 1? This is strange. * Line 972: how do you get the factors 4 and the 2, this looks like Young's inequality but not clear at all.

Rating

6

Confidence

4

Soundness

3

Presentation

2

Contribution

2

Limitations

N/A

Reviewer b9A47/10 · confidence 5/52024-07-09

Summary

This paper investigates the issue of dual drift caused by the mismatch between primal and dual variables when using partially participated training in federated learning (FL). It proposes a novel method, A-FedPD, which employs virtual dual updates to mitigate these negative impacts. Comprehensive theoretical analysis and extensive experiments are presented to verify the effectiveness of the proposed method.

Strengths

1. The experiment in Figure 1 vividly illustrates the source of defects in primal dual methods under partial participation. 2. The study presented in this paper is novel. Federated primal dual methods have long exhibited varying degrees of instability in experiments. The analysis based on dual drift provided in this paper is robust and useful. 3. The quadratic term in the primal dual method is used in this paper to obtain a reduced form of the stability bound in the iterative process, ultimately achieving a constant bound in Equation (10). This demonstrates that the proposed method is superior to the federated averaging method in terms of generalization under the same training process. This conclusion is broad and can be extended to other primal dual methods. 4. The experiments were solid and comprehensive in their analysis. In terms of both communication efficiency and wall-clock time testing, the A-FedPD method shows strong performance.

Weaknesses

1. What is the variant A-FedPDSAM? I didn’t see an introduction to this algorithm in the text; please remind me if I missed this part. 2. The performance of the algorithm is improved by trading space for time. During a global update, storing the dual variables requires considerable resources. Although federated learning primarily considers communication bottlenecks, this still increases server-side consumption. 3. I noticed that the FedDyn column in the main table indicates failure. Were all hyperparameters tested in the experiment? There should be an additional discussion to clarify whether this failure is a special case or not.

Questions

The same with the weaknesses part.

Rating

7

Confidence

5

Soundness

3

Presentation

3

Contribution

3

Limitations

The same with the weaknesses part.

Reviewer YYQB5/10 · confidence 3/52024-07-10

Summary

A primal-dual-based federated learning algorithm (A-FedPD) is proposed to mitigate the drift of local dual variables. In federated learning with partial participation training, the local dual variables in inactive clients can be drifted. To mitigate this issue, the propose A-FedPD would be effective since the local dual variables in unparticipated clients are aligned with the global average models. Through extensive experiments, including nonconvex neural network models, the effectiveness of the A-FedPD was empirically investigated.

Strengths

(S1) Simple but effective method: It is well known that the local dual variables in unparticipated clients can be drifted in the primal-dual based federated learning algorithms. The idea of modifying the unparticipated local dual variables to bring them closer to the global model is simple but effective. (S2) The proposed Algorithm 1 has been theoretically analyzed under certain assumptions, demonstrating a near-optimal convergence rate. One point of dissatisfaction is the comparison of convergence rates with several primal-dual based methods that will not be affected by dual drift in Table 7. Such an important results are to be included in the main paper. (S3) The effectiveness of the proposed method has been validated through several image classification benchmark tests using neural network models. Its robustness against various stresses (participation ratio, local interval, data heterogeneity) has also been empirically investigated.

Weaknesses

(W1) In the experiments of Sec. 5.2, there seems to be no significant difference between the proposed method FEDPD and the comparison method FEDSPEED. The reviewer has serious concerns about this point. Without introducing SAM, can the superiority not be empirically demonstrated? (W2) In addition, the consideration of comparison methods in Sec. 5.2 seems insufficient. FEDSPEED performs better, but why is this? It also compares with primal methods such as SCAFFOLD and FedSAM, but why is the proposed method better than these methods? (W3) Some relevant literature might be missing. [a] R. Pathak et al., “FedSplit: An algorithmic framework for fast federated optimization,” NeurIPS 2020. [b] G. Zhang et al., “Revisiting the Primal-Dual Method of Multipliers for Optimisation Over Centralised Networks”, IEEE Transaction on Signal and Information Processing, pp. 228-243, 2022.

Questions

(Q1) Could you provide proof sketch for theoretical analysis in Sec. 4? I would particularly like to know about the novel points in comparison with analyses in prior studies, such as FedPD. (Q2) On line 309, it is written that “we freeze most of the hyperparameters for all methods,” however these were searched in the list written in the Grid Search column of Table 3, right? I think it would be better to clearly write in the main paper that a hyperparameter search was conducted (see Appendix). (Q3) In Figure 2(b), it seems that accuracy is lower when the local interval is short. Does the total number of updates differ according to the local interval?

Rating

5

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

Limitations are noted in the Appendix. Social impact is not mentioned.

Reviewer 6Q4p7/10 · confidence 4/52024-07-12

Summary

This paper studies the primal-dual-based FL algorithm with partial client participation. The inactiveness of the local clients causes both local primal and dual variables to drift from their expected value and slows down FedPD's convergence. This paper provides a fix to the FedPD algorithm in the partial participation setting by moving the dual update to the server side and allowing the server to store the dual variables. The theoretical result shows that A-FedPD can achieve the same convergence rate as FedPD ($O(1/T)$), and the error bound is better than the prima-only FL algorithm. Numerical results also show that the proposed algorithm achieves the best accuracy in different settings.

Strengths

1. Soundness: the theoretical analysis provides a clear convergence and generalization result for the proposed algorithm. Numerical results also show that the proposed algorithm outperforms the existing algorithms on different models and data distributions. 2. Clarity: the paper is clearly written with adequate reference to prior works and clear notations and results.

Weaknesses

1. Memory inefficiency: The algorithm requires saving the local dual variable of all clients. When C is large, this might cause a large memory cost on the server, especially in the FL setting. This might restrict the proposed algorithm's use case. This is discussed in the limitations.

Questions

How would A-FedPD (and other algorithms) perform when the communication patterns of different clients are different, i.e., $\mathbb{E}_P[\theta_i] \neq \bar{\theta}$?

Rating

7

Confidence

4

Soundness

3

Presentation

4

Contribution

3

Limitations

N/A

Reviewer 6Q4p2024-08-07

Thanks for the response. I don't have any further questions.

Authorsrebuttal2024-08-08

Thanks for the review

Thank you again for the time and effort paid for reviewing our submission.

Reviewer b9A42024-08-08

Thanks for the authors' response. I have no additional questions concerning this paper.

Authorsrebuttal2024-08-08

Thanks for the review

Thank you again for the time and effort paid for reviewing our submission. We will revise the corresponding content based on the summary in the rebuttal.

Reviewer c2hH2024-08-08

Papers with strong theoretical contributions and papers with heuristics showing strong empirical performance are both valuable. Also, nonconvex optimization is more difficult than convex optimization. I understand that you propose a heuristic technique to mitigate the negative effects of partial participation in nonconvex federated learning. You motivate your technique by an intuition based on correcting the "dual drift". My negative evaluation is based on the fact that there is a theoretically-grounded literature for the convex case, and it seems to contradict your intuition that it is bad if the dual variables of idle clients remain unchanged for a long time. In other words, your technique should be shown to work in the "simple" convex case, which is not the case. How do we know that A-FedPD will not diverge in even simple quadratic synthetic experiments? Showing improved generalization efficiency in some practical examples is fine, but this is not enough to make progress in our understanding of primal-dual optimization methods for federated learning, as the title claims, in my opinion. At this time, I am keeping my score.

Authorsrebuttal2024-08-08

Rebuttal from authors

Thank you for your positive response and we are honored and happy to continue the discussion with you. Regarding your comments above, we have listed and addressed the two concerns you raised below. ## About the comment "there is a theoretically-grounded literature for the convex case seems to contradict your intuition". In our overall response, we have made it very clear that even for non-convex objectives, the classical federated primal-dual method FedADMM with partial participation have been theoretically proven to converge [X1]. However, despite the theoretical guarantees, our experiments have validated that FedADMM still faces extreme instability and even divergence during partial participation training (our figure 1). Obviously, **pure theoretical analysis can not provide an absolute guarantee** of an algorithm's feasibility. Therefore, it is not appropriate to dismiss the existence of the problem simply because it has been proven in theory. In fact, we are not the first to observe such non-convergence phenomena in experiments; previous classical works [X2,X3,X4] have repeatedly demonstrated the existence of this phenomenon experimentally. [X4] also indicates that the update mismatch between primal and dual variables leads to a "drift", which is related to the ``dual drift" we pointed out in this paper. [X1] Wang H, Marella S, Anderson J. Fedadmm: A federated primal-dual algorithm allowing partial participation[C]//2022 IEEE 61st Conference on Decision and Control (CDC). IEEE, 2022: 287-294. [X2] Xu J, Wang S, Wang L, et al. Fedcm: Federated learning with client-level momentum[J]. arXiv preprint arXiv:2106.10874, 2021. [X3] Baumgart G A, Shin J, Payani A, et al. Not All Federated Learning Algorithms Are Created Equal: A Performance Evaluation Study[J]. arXiv preprint arXiv:2403.17287, 2024. [X4] Kang H, Kim M, Lee B, et al. FedAND: Federated Learning Exploiting Consensus ADMM by Nulling Drift[J]. IEEE Transactions on Industrial Informatics, 2024. ## About the comment "Your technique should be shown to work in the "simple" convex case". We cannot agree with this comment. As you mentioned above, nonconvex optimization is more difficult than convex optimization. In fact, **a large number of papers on learning federated primal-dual methods conduct the experiments on non-convex experiments [X1, X2, X3, X4, X5]**. More importantly, **the "dual drift" issue is discovered in non-convex experiments** by several previous works (mentioned in the answer above), and our technique is aimed at addressing this problem. We have not identified this issue in convex experiments. Therefore, the reviewers have no reason to ask us to implement this work for convex objectives. If the ``dual drift" issue does not indeed exist in convex experiments, then our virtual update technique is not necessary for training convex models. However, since a lot of studies have identified this problem in non-convex experiments, we have validated that our method effectively addresses this issue in non-convex experiments. [X1] Zhang X, Hong M, Dhople S, et al. Fedpd: A federated learning framework with adaptivity to non-iid data[J]. IEEE Transactions on Signal Processing, 2021, 69: 6055-6070. [X2] Acar D A E, Zhao Y, Matas R, et al. Federated Learning Based on Dynamic Regularization[C]//International Conference on Learning Representations. [X3] Sun Y, Shen L, Huang T, et al. FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracy[C]//The Eleventh International Conference on Learning Representations. [X4] Kang H, Kim M, Lee B, et al. FedAND: Federated Learning Exploiting Consensus ADMM by Nulling Drift[J]. IEEE Transactions on Industrial Informatics, 2024. [X5] Zhang Y, Tang D. A differential privacy federated learning framework for accelerating convergence[C]//2022 18th International Conference on Computational Intelligence and Security (CIS). IEEE, 2022: 122-126. We noted that the reviewers show concerns about the naming. One is with the term "dual drift", and the other is with the title. We believe these are very easy to resolve. We would greatly appreciate any suggestions the reviewers might have for the names.

Reviewer c2hH2024-08-09

I am happy to participate to this polite and argued discussion. 1) I don't understand your reply. If FedADMM is proved to converge (in nonconvex problems with partial participation), it is not possible that it diverges in practice (unless too agressive stepsizes are chosen, but this is normal). 2) I see, this is interesting if the dual drift issue exists in the nonconvex setting but not in the convex setting. I did not request you to study the convex setting, I was just not convinced by the dual drift issue that you are considering in the first place, precisely because I am familiar with the convex setting and did not observe this issue. To acknowledge the merits of your study, I am increasing my score to 6.

Authorsrebuttal2024-08-09

Thanks for the review

**We first want to express our sincere gratitude for your kind ongoing discussion. From the review and discussion, we are also confident that your research in the area of convex problems is very profound.** We are honored to share some of the issues we are currently encountering in the non-convex experiments and the inconsistencies with the theory. Before the discussion stage, we carefully learned the series of papers you mentioned above. At the same time, we observed an issue: the studies in convex objective research are precise. Both the accuracy of local Lagrangian function solutions and the constants in the assumptions can be meticulously controlled. However, it is challenging to achieve effective control in experiments with non-convex objectives, especially as the models become more complex and the datasets grow larger. Therefore, there is a gap between theory and practice. This is why FedADMM has been theoretically proven to achieve partial participation, but still faces significant issues in practice, they have not rigorously verified whether the experiments can satisfy the assumptions and conditions, and how to satisfy them. This is also a challenge and difficulty in the current field of research on non-convex problems. In this paper, our motivation comes from extensive experimental validation. We found that divergence never occurs in the case of full participation. It only happens with partial participation, and the divergence becomes more severe as the proportion of partial participation decreases. We understand your concern: "unless too aggressive step sizes are chosen, but this is normal." We have explored nearly all possible hyperparameters to find stable convergence scenarios (as shown in the appendix). Unfortunately, divergence at low participation rates is difficult to control. Combined with the descriptions of this experimental phenomenon in many previous works, we are confident that there is an important issue waiting to be addressed here. This also inspired us to propose a technique that makes partial participation resemble full participation as closely as possible. **We appreciate your recognition of our paper.** We will revise the paper according to the results of our discussions: (1) adding an introduction to the theoretical advances of federated dual methods in section related work; (2) correcting the $O(L^3)$ notation; (3) fixing all typos; (4) incorporating some paragraphs from the discussion into the main text to further clarify the situations in which the problem arises. **Discussion with you has been very helpful and has made the paper more complete. We thank you again for your time and effort in reviewing this paper. If you have any further questions, we are very happy to continue discussing these academic issues with you and learn from you.**

Reviewer YYQB2024-08-09

W1: I clearly understand your explanation. However, I couldn't extract this claim from the original version. Could you improve the experimental section to explain this better? W2: Including this table that explains the relationships between the methods in the Appendix would help enhance the readers' understanding. Q1: Adding this to the Appendix would make it easier to understand the contributions in the theoretical analysis. I do not have any questions furthermore. I keep my score.

Authorsrebuttal2024-08-09

Thanks for the review

We are very grateful for your review of our rebuttal and are pleased that all your concerns have been addressed. We greatly appreciate your suggestions. We will add a horizontal table above Table 2 in our paper to categorize each method, and add the explanations in the rebuttal above regarding optimization and generalization techniques in the appendix. Thank you again for your recognition of our work.

Program Chairsdecision2024-09-25

Decision

Accept (spotlight)

© 2026 NYSGPT2525 LLC