Momentum Provably Improves Error Feedback!

Due to the high communication overhead when training machine learning models in a distributed environment, modern algorithms invariably rely on lossy communication compression. However, when untreated, the errors caused by compression propagate, and can lead to severely unstable behavior, including exponential divergence. Almost a decade ago, Seide et al [2014] proposed an error feedback (EF) mechanism, which we refer to as EF14, as an immensely effective heuristic for mitigating this issue. However, despite steady algorithmic and theoretical advances in the EF field in the last decade, our understanding is far from complete. In this work we address one of the most pressing issues. In particular, in the canonical nonconvex setting, all known variants of EF rely on very large batch sizes to converge, which can be prohibitive in practice. We propose a surprisingly simple fix which removes this issue both theoretically, and in practice: the application of Polyak's momentum to the latest incarnation of EF due to Richtárik et al. [2021] known as EF21. Our algorithm, for which we coin the name EF21-SGDM, improves the communication and sample complexities of previous error feedback algorithms under standard smoothness and bounded variance assumptions, and does not require any further strong assumptions such as bounded gradient dissimilarity. Moreover, we propose a double momentum version of our method that improves the complexities even further. Our proof seems to be novel even when compression is removed from the method, and as such, our proof technique is of independent interest in the study of nonconvex stochastic optimization enriched with Polyak's momentum.

Paper

References (88)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer naHZ6/10 · confidence 4/52023-07-05

Summary

This paper introduces a modification to the EF21-SGD algorithm by incorporating momentum, resulting in a new algorithm named EF21-SGDM. The innovative analysis accompanying this new method successfully addresses the challenges associated with EF21-SGD, reducing the sample complexity from $\Omega(\sigma^2/\epsilon^2)$ to $\mathcal{O}(\sigma^2/(L\delta_0)$. Importantly, EF21-SGDM operates without requiring an assumption of bounded gradient.

Strengths

The paper proposes a new algorithm for distributed settings with compressed gradient. The authors also provide an original analysis that delivers improved results, contributing substantially to the existing work.

Weaknesses

Although the paper's sample complexity in each iteration number is independent of $\varepsilon$, it still depends on the variance term, $\sigma$. This dependence should be explicitly stated to ensure a comprehensive understanding of the algorithm. Furthermore, Algorithm 1 uses $B_{init}$, but the batch size remains consistent throughout the iterations. It may be more appropriate to avoid this term, considering the batch size does not vary.

Questions

The theoretical results suggest that $\eta=\mathcal{O}(1/T)$, but the experiment utilizes $\eta=0.1$. Could the authors clarify the reasoning behind this discrepancy between theoretical and experimental parameters?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

No.

Reviewer 5FHY7/10 · confidence 4/52023-07-06

Summary

This paper proves that momentum helps EF21-SGD. It makes several non-trivial contributions. First, it theoretically shows that EF21-SGD cannot converge when batch-size is small. Second, it proposes a simple remedy to this issue, i.e., incorporating momentum to EF21-SGD. Third, it proves that EF21-SGDM can converge well even if with small batch-size. Fourth, it extends EF21-SGDM can achieve linear speedup in distributed non-convex optimization without any assumptions on bounded gradient similarity. All these contributions are solid and novel.

Strengths

- It theoretically shows that EF21-SGD cannot converge when batch-size is small. - It proposes a simple remedy to this issue, i.e., incorporating momentum to EF21-SGD. - It proves that EF21-SGDM can converge well even if with small batch-size. - It extends EF21-SGDM can achieve linear speedup in distributed non-convex optimization without any assumptions on bounded gradient similarity. - The paper is well-written and easy to follow.

Weaknesses

The paper is well-written. I have a few minor questions. 1. In table 1, the authors claim that NEOLITHIC uses a large mini-batch, which may not be correct. While NEOLITHIC uses R times larger batch-size than EF21-SGD per iteration, it runs R times fewer iterations than EF21-SGD (see the NEOLITHIC algorithm in Huang et. al., 2022). On average, NEOLITHIC uses a normal O(1) mini-batch as EF21-SGD. 2. Typo: In line 176, "such" should be "such as". 3. In line 182, the authors claim that the proof technique can help establish linear speedup for Scaffold without relying on data similarity assumption. But Scaffold is not relying on this assumption, right? 4. In Figure 2, NEOLITHIC is far slower than EF21-SGD, which somehow contradicts with the results shown in Huang et. al., 2022. Can the authors provide more details on the experimental settings on NEOLITHIC? How many iterations do NEOLITHIC run? Is accumulated gradient used? How does the total communication cost be counted in NEOLITHIC? Does NEOLITHIC converge slower than the other algorithms in terms of iterations (not in communication cost)?

Questions

See above.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

N/A

Reviewer hi3u5/10 · confidence 3/52023-07-06

Summary

The authors present a new method called EF21-SGDM by combining EF21 and Polyak's momentum SGD. The theoretical contribution is that , it improves the communication and sample complexities of previous error feedback algorithms under standard smoothness and bounded variance assumptions. They also propose a double momentum variant to further improve the complexity. The experiments are conducted with non-convex logistic regression problems.

Strengths

- only standard smoothness and bounded variance assumptions are needed. - identify an issue that EF21 with stochastic gradients has weak sample complexity guarantees, and fix it by leveraging our new Lyapunov function construction and new analysis. - sample complexity is free of $\alpha$ and batch-free, the best from table 1. - The theoretical analysis is comprehensive and seems solid.

Weaknesses

- Compared with theoretical contribution, the algorithm itself is straightforward, i.e., combining two existing methods EF21 and SGDM. - The experiments are validated on only logistic regression, where large-scale distributed training is not as crucial as in larger models. - EF21-SGD2M is not implemented in experiments.

Questions

Does EF21-SGD2M has practical benefits over EF21-SGDM? If not, how do you verify the complexity is improved over EF21-SGDM?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The authors did not discuss societal impact. I believe this is a theoretical paper and do not see much negative societal impact.

Reviewer Pi6q4/10 · confidence 3/52023-07-07

Summary

The authors propose a new version of EF-SGD which uses momentum. The authors show that under standard assumptions the proposed method has a better convergence rate. The authors claim that in several cases it is hard to perform large batch sampling like when performing RL training. To overcome this problem they propose a momentum based method, which can overcome the small batch issue. The perform additional experiments on Mnist and real sim datasets and compare there effectiveness.

Strengths

1. The proof indeed looks novel the assumptions are reasonable and the improvement in rates for small batch size is very encouraging. 2. The problem is well motivated. 3. The authors perform experiments and compare with different version of EF-SGD.

Weaknesses

1. EF21 SGD although theoretically motivated is not a practical due to only 1 way compression and often highoverhead, authors should comment on the real world implication of EF21-SGD and EF21-SGDM. 2. The authors are motivating their problem using examples from Medical Literature and Federated RL, but the actual experiments are performed on MNIST. In 2023, work like https://www.mosaicml.com/blog/mosaic-resnet trains Imagenet in 27 minutes using just 8 GPU, shouldn’t the paper at least have experiments using Cifar. 3. The only comparison is with other EF21-SGD variants, can you please have additional comparison with other communication efficient methods. And the comparisons should be on wall clock time rather than bits communicated. 4. Authors seems to be not accounting the memory consumption needed because of momentum, it would be great to have a discussion on that.

Questions

Please comment on the concerns raised in weakness sections.

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

3 good

Contribution

2 fair

Limitations

The authors in the opinion of this reviewer have not addressed limitations of their methods and their applicability to problems.

Reviewer Pi6q2023-08-13

Additional Clarifications

1. I am curious why did the authors turn off all the optimizations to achieve SOTA accuracies ? These optimizations like LR decay, data augmentation are standard and are widely used. 2. "A theorem is worth a thousand experiments" and "Our paper is not a systems/software work, where comparison using run time would be appropriate and expected" unfortunately I as a reviewer strongly disagree with these statements. However, I understand the authors have a different opinion. I personally believe to have a meaningful impact both theoretical and experimental validation should be provided on real metrics. For example - Work like Powersgd (Vogels et al. ) which end up having meaningful impact on improving accuracy had extensive experiments. I understand from the statements that this might not be goal of reviewers, but for a top tier conference like Neurips it would be expected to provide real world impact. Especially in the case where previous methods have been actually compared on runtime. Given these issues I can not convincingly recommend for the acceptance of the paper.

Authorsrebuttal2023-08-14

Re: Additional Clarifications

> I am curious why did the authors turn off all the optimizations to achieve SOTA accuracies ? These optimizations like LR decay, data augmentation are standard and are widely used. We do this for several reasons: - First, our paper's aim is *not* to compete with methods/papers/systems whose goal is to achieve SOTA generalization accuracies on selected benchmarks - we agree that in such work such optimizations should be used. Our work is of a completely different variety : our paper is not about generalization at all. Our work isolates an open theoretical question (can error feedback provably work with small minibatches?) and proposes an algorithmic fix (use of momentum), and theory which conclusively shows that this trick works as advertised, in the class of smooth nonconvex functions. - Second, our experiments are designed to test the predictive power of our theory. Heuristics such as data augmentation and LR decay are orthogonal considerations which are entirely irrelevant in our study. They are important as far as actual generalization performance of various optimizers is concerned, but as explained above, this is not the subject of our paper. For this reason, if we included these heuristics in our experiments, it would actually make the experiments and conclusions one can draw from them more confusing. > "A theorem is worth a thousand experiments" and "Our paper is not a systems/software work, where comparison using run time would be appropriate and expected". Unfortunately I as a reviewer strongly disagree with these statements. However, I understand the authors have a different opinion. Yes, we are of a different opinion. We strongly believe that theory and empirics have equal value in ML research. One feeds into the other and vice versa. We believe that the ML field needs to stand on both its feet (theory and empirics) to advance and to be truly useful. State of the art empirics typically stands on the shoulders of strong theory, and uses additional tricks, heuristics and ideas to push things further. Such tricks are then studied by theoreticians, uncovering their robustness or brittleness, improving and modifying them, or replacing them with more theoretically well grounded tricks. The actual interplay between theory and practice is much more complicated and intricate than this, of course. Once you have the belief that theory and empirics have equal intrinsic value (and we actually believe this is what the community ideally/hopefully *should* believe), then it becomes clear that the community should be able to equally appreciate strong theory and empirical works. Fortunately, as the record of papers accepted to NeurIPS in the past clearly shows, this is the case. We believe it is in fact dangerous to use a double standard. For example, - We believe that a strong practical/empirical work should stand on its own, and be perfectly acceptable to NeurIPS without the requirement that it contains any theory whatsoever. Of course, some theory is welcome, and can make that paper even stronger (say an 8/9/10), but it should not be a requirement for acceptance. - Likewise, a strong theory work should stand on its own, and be perfectly acceptable to NeurIPS without the requirement that it contains any experiments whatsoever. Of course, some empirics is welcome, and can make that paper even stronger (say an 8/9/10), but it should not be a requirement for acceptance. It seems to us you do not subscribe to this philosophy. > I personally believe to have a meaningful impact both theoretical and experimental validation should be provided on real metrics. This can't possibly be the way to evaluate NeurIPS papers since otherwise no pure empirical and pure theory work would ever get published - and there are many examples of immensely influential works in these categories. > For example - Work like Powersgd (Vogels et al. ) which end up having meaningful impact on improving accuracy had extensive experiments. This paper is not a theory paper - it does not include a single theorem in the main body of the paper. The one theorem in the appendix is minor, and not central to the paper. This is a very good example of an empirical work which we believe should be accepted to a top venue. In the same manner, pure theory papers with perhaps just one experiment in the appendix should also be perfectly acceptable to a top conference. As an example, consider the 2012 NeurIPS paper by Nicolas Roux, Mark Schmidt and Francis Bach: A Stochastic Gradient Method with an Exponential Convergence Rate for Finite Training Sets. This work is of a theoretical nature just like ours, isolating an important theory problem and proposing a solution. The experiments are designed to test the theory. Yet, this work won the Lagrange Prize in Continuous Optimization, and had an enormous impact.

Authorsrebuttal2023-08-14

Re: Additional Clarifications (part 2)

> I understand from the statements that this might not be goal of reviewers, but for a top tier conference like Neurips it would be expected to provide real world impact. Especially in the case where previous methods have been actually compared on runtime. We disagree. We believe that each work needs to be judged based on its own merits, and by the standards of the subfield/field it belongs to. Theory works need to judged based on the theoretical breakthroughs and contributions, works that build systems should be judged on the real world efficiency of the systems, network architecture works on the benefits the architecture brings, and so on. If we use a single parameter to judge all works (e.g., SOTA generalization performance), we would be doing a massive disservice to the community, and would effectively narrow down the scope of the field to the detriment of of everybody. NeurIPS papers are like the olympics. Different fields have different quality standards. We can't judge all sports by the standard of one. We can't evaluate a marathon runner by the standards of 100m sprint. If we did so, we would disqualify marathon as a discipline, and would not be able to appreciate even a world-record-breaking marathon run.

Reviewer naHZ2023-08-18

Thank you for the rebuttals. I do not have more questions.

Reviewer 5FHY2023-08-21

Thanks for the rebuttal

I thank the authors for the rebuttal. They have addressed my concerns. I will keep my current rating and vote to accept the paper in the discussion. Please clarify in the camera-ready version that your experimental results are not contradictory with [Huang et. al. 2022] because of the choices of the $R$ value.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC