Memory-Efficient LLM Training with Online Subspace Descent

Recently, a wide range of memory-efficient LLM training algorithms have gained substantial popularity. These methods leverage the low-rank structure of gradients to project optimizer states into a subspace using projection matrix found by singular value decomposition (SVD). However, convergence of these algorithms is highly dependent on the update rules of their projection matrix. In this work, we provide the \emph{first} convergence guarantee for arbitrary update rules of projection matrix. This guarantee is generally applicable to optimizers that can be analyzed with Hamiltonian Descent, including most common ones, such as LION, Adam. Inspired by our theoretical understanding, we propose Online Subspace Descent, a new family of subspace descent optimizer without SVD. Instead of updating the projection matrix with eigenvectors, Online Subspace Descent updates the projection matrix with online PCA. Online Subspace Descent is flexible and introduces only minimum overhead to training. We show that for the task of pretraining LLaMA models ranging from 60M to 7B parameters on the C4 dataset, Online Subspace Descent achieves lower perplexity and better downstream tasks performance than state-of-the-art low-rank training methods across different settings and narrows the gap with full-rank baselines.

Paper

References (24)

Scroll for more · 12 remaining

Similar papers

Peer review

Reviewer FtMv5/10 · confidence 3/52024-06-29

Summary

The paper considers memory-efficient optimization for large language model. In particular, the focus is on optimizers leveraging low-rank projection. The paper first derives asymptotic convergence for such methods with arbitrary choice of projection matrix. Then the paper identifies the inefficiency of using SVD to construct the projection matrix and proposes online subspace descent method by updating the projection matrix with online PCA. Experiments show such method improves the efficiency of training LLM while achieving comparable performance compared to low-rank baseline with SVD projection.

Strengths

1. The paper derives the first convergence result for optimizers with low-rank projection. 2. The paper addresses the inefficiency of using SVD to construct the projection matrix by online subspace descent. 3. Experiments show the benefits of the online subspace descent over the SVD baseline.

Weaknesses

1. The main downside of the proposed method is that it could potentially increase the memory overhead. It seems to me that the idea of online updating $P_t$ trades off computation with memory. Although it reduces the computational cost of not computing SVD, it nevertheless increases the memory especially using Adam type of optimizer for $P_t$. In Line 216, the paper states the memory overhead is 9.01GB versus 8.64GB. However, it is not clear what model this numbers refer to. I suspect with the increase in rank and model size, the memory overhead could become larger. 2. The use of Hamiltonian descent and Lyapunov analysis for showing convergence is not properly motivated. Because the main result (Theorem 4.5) only shows asymptotic convergence to stationary point of $L(W)$, it is not convinced why to derive such results under continuous time limit. Can such result also be derived in the discrete case and can non-asymptotic result be derived? 3. Figure 2 is not a fair comparison because in practice, SVD is not used every update.

Questions

1. In Line 233, what does it mean to be high-frequency versus low-frequency tokens? And is there any verification or justification that high-frequency tokens are learned with low-rank training while learning low-frequency tokens requires higher ranks? 2. In Line 252, can you comment further why AdamW is more suitable for online PCA?

Rating

5

Confidence

3

Soundness

2

Presentation

3

Contribution

2

Limitations

The paper has discussed limitations.

Authorsrebuttal2024-08-11

The derivation presented above provides a foundational outline for our convergence analysis. Building on this, we can conduct a more detailed analysis following the established classical (or traditional) approach. Taking a step further from the bound above, we have $(1 - \frac{L \epsilon}{2}) \frac{1}{T} \sum_{t=0}^{T-1} ||\hat{M}_t||_2^2 \leq \frac{H(W_0, \hat{M}_0) - H(W_T, \hat{M}_T)}{\epsilon T}$ $+\epsilon \frac{1}{T} \sum_{t=0}^{T-1} \alpha ||\hat{G}_t - \hat{M}_t||_2^2$ One can use the established standard methods to bound the last term $||\hat{G}_t - \hat{M}_t||_2^2$ (for example at page 11-12 in the supplementary from [Sun et al. 2023]). ### References Tao Sun, Qingsong Wang, Dongsheng Li, and Bao Wang. Momentum ensures convergence of signsgd under weaker assumptions. In *International Conference on Machine Learning*, pages 33077–33099. PMLR, 2023.

Reviewer RBsr5/10 · confidence 3/52024-07-12

Summary

The authors propose a variation of GaLore that doesn't require a fixed SVD for achieving a memory efficient LLM training. Instead, the authors project the gradients dynamically into a small sub-space using online PCA which depends on the evolving trace of the gradient landscape. The authors tested their method against GaLore using perplexity as a metric while training on the C4 dataset.

Strengths

- the paper is well written and easy to follow - having a dynamically changing projection matrix based on the gradient landscape makes total sense - the theoretical explanation shows an interesting perspective on why this proposed method of using online pca for matrix projection could work - the results is promising in C4 as the memory seems more or less similar in efficiency as GaLore with similar perplexity.

Weaknesses

- the perplexity results is not enough to show that the method works well on standard NLP tasks like GLUE which consists of the datasets MRPC, COLA, STS-B, RTE, SST2, MNLI, QNLI, QQP to see if the actual downstream task performance is still retained with the proposed projection method. - while the theory is interesting, it seems to include many strong assumptions that do not reflect real-world datasets - the memory savings and the perplexity is very similar to GaLore and it is not clear that it is significant. The authors need to do multiple runs and compute the standard deviation to make the results are not due to noise.

Questions

please answer how you could address the weaknesses above

Rating

5

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

Yes

Reviewer 6zai6/10 · confidence 2/52024-07-13

Summary

Utilizing low-rank structure has become a popular way for memory-efficient LLM training. The authors are the first to provide convergence guarantees for general low-rank updating rules. Furthermore, based on their theoretical result, they propose a family of optimizers called online subspace descent. The empirical result shows that their performance is better than existing low-rank training methods.

Strengths

This work provides a general theoretical guarantee for a range of low-rank updating rules, which can guide and help future research in this direction.

Weaknesses

While the authors provide mathematical proofs, it could be more beneficial to the readers if they can provide more high-level intuitions why these methods could converge. As stated by the authors, the convergence result is surprising under the complications.

Questions

Could you provide more high-level intuitions on why the methods could converge under the complications?

Rating

6

Confidence

2

Soundness

3

Presentation

3

Contribution

3

Limitations

Yes, the authors discuss the limitations.

Reviewer NLbB7/10 · confidence 4/52024-07-14

Summary

The paper presents Online Subspace Descent, a memory-efficient modification applicable to a wide class of gradient-descent based algorithms where low-rank projections of gradients can be employed to reduce the memory overhead. Contrary to recent techniques such as GaLore, which require infrequent but costly updates of the projection matrix, the proposed method updates the projection matrix in an on-line manner, and proves the convergence of the resulting algorithm.

Strengths

Memory-efficient optimizers are crucial to enable LLM training/fine-tuning outside of huge datacenters; as such, this is a potential high-significance paper. While I think the idea of gradually updating the projection matrix is pretty much "the obvious thing to do", I really like that the authors didn't just propose "Adam[Add-some-random-letter-here]", but instead took a step back and showed that the same principle applies to a wide class of optimization algorithms.

Weaknesses

* There are some grammatical errors and typos, I think (not a native speaker), but not to the degree that they hamper the understanding of the text (e.g., I think it should be: update rules of _the_ projection matrix; in the subspaces that dynamically _change_, How Should P be _updated_, is not a problem for optimizers that _yield_) l.1ß6: `Note that the update of P t can be done in parallel with that of 106 (W t, ˆ St), and incurs no slow down once it is fast enough to not cause a speed bottleneck` Generally, Adam updates are memory-bound operations, so I'm not sure if you can schedule a second operation concurrently without slowing down the update. Also, the same could be said for GaLore, you run you SVD in parallel to the main network; if it takes multiple steps, you could just wait till it is done, and update only afterwards, so GaLore can be made as parallel as the algorithm that is proposed here, I think. l.119: `In this work, we propose to update P t in a continuous online fashion that incorporates the most 119 recent gradient information in a timely fashion, without calling linear algebra routines.` Matrix multiplications are linear algebra routines, so I think most P updates will involve some linear algebra. I think the description around l. 119-128 could do with some rewriting. The sentence `we propose to update P t at time t by minimizing the following PCA objective` is misleading, because P_t is not actually updated like that; instead P_t is one step along some optimizer that decreases this objective. Second, `Instead of minimizing [...] exactly, to retain computational efficiency, we propose` seems to be misleading, too; in an earlier paragraph, it was GaLore was criticized for using only a single gradient information for its update. This sentence suggests that if you had enough compute, you would do the same here, only due to computational constraints do you end up with something that integrates multiple gradients, which I think goes against the message you want to make in the paper. l. 170: `There is no requirement on Γ here, besides that the derivative in (8) should exist` Γ does not appear in (8), so it isn't clear which derivative should exist. l. 223: `The typical Pytorch implementation of SVD can be up to 142 times slower than running a single step online PCA on representative weight tensors from LLaMA architectures.` Given that GaLore runs SVD only every 300 steps, this would make GaLore comutationally more efficient than the proposed method.

Questions

It has been observed that scaling up LLMs is notrivial also from a numerical stability point of view, e.g., in quantization, starting with 7B models, outliers become much more of a problem than they are for smaller models. As such, it would be great to see this method applied to larger models, at least to 7B, ideally even larger. If compute is a problem, maybe scaling the amount of training steps less than would be optimal could help, to at least have some proof-of-concept for the method at larger model scales. l.145: `this is a mild condition that can be satisfied e.g., when P_t is updated by (online) PCA on G_t` Is this really true, in the generality that is suggested here? If so, I'd like to see a formal argument. The paper only fixes the objective eq. (6), but leaves the choice of optimizes to the user, and my intuition says it should be possible to construct an example where minimizing (6) with a momentum-based optimizer could lead to $<G_t, P_t>$ being zero without $G_t$ being zero. l 158, I'm not sure what "once" is supposed to mean here. Should it just be "if"? Regarding the convergence guarantee for arbitrary $P_t$, I think it would be illuminating to consider what happens in case of an adversarial choice; for example, for SGD with momentum, choose $P_t = min <G_t, P_t^T P M_t>$, i.e., choose P so that it projects the momentum buffer _against_ the direction of the current gradient. Why does this choice of P not break convergence? l. 235 > In conclusion, given sufficient time and resources, higher ranks yield better performance for Online Subspace Descent. It is recommended to select the highest rank until the perplexity reduction saturates. Can this reliably be selected a priori, i.e., does picking a rank optional for running optimization for, e.g., 10k steps, also result in optimal rank for training until "convergence"

Rating

7

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

Evaluations for production-size LLMs are missing.

Reviewer FtMv2024-08-11

Thank you for the responses. For the discrete-time convergence analysis, first, the convergence is measured in terms of gradient momentum, which is non-standard. Second, there seems to be an extra term $\epsilon B_t$, which depends on the entire past history as well as the gradient momentum, which is also on the left hand side of the final inequality. Can the authors clarify?

Reviewer 6zai2024-08-11

Thank you for the response to my question!

Reviewer NLbB2024-08-13

Thank you for the response, which adequately addresses the concerns raised in my review. With the additional clarifications as described, and an additional experiment in the regime where memory-efficient optimizers start to become very important and useful, I think this is a good paper that, were I to see it for the first time at the conference, I probably would point out to my colleagues. I will raise my score to an "Accept" rating.

Reviewer RBsr2024-08-14

Thank you for the rebuttal and for addressing most of my concerns about the assumptions and memory saving. I have increased my score by 1.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC