DropBP: Accelerating Fine-Tuning of Large Language Models by Dropping Backward Propagation

Summary

The paper introduces DropBP, an innovative approach to accelerate the fine-tuning of Large Language Models (LLMs) by selectively dropping layers during backward propagation. This method is presented as a means to reduce computational costs and activation memory, significant challenges in the efficient fine-tuning of LLMs. The authors have provided a clear implementation of DropBP as a PyTorch extension and demonstrated its effectiveness through experiments on various LLMs and datasets.

Strengths

- The concept of dropping backward propagation layers to reduce computational overhead is differential from previous work and addresses an important issue in training large models. - The paper includes extensive experiments that validate the effectiveness of DropBP in reducing training time and memory usage while maintaining accuracy. - The development of a PyTorch extension for DropBP facilitates easy integration with existing training codes, enhancing the practical applicability of the method.

Weaknesses

- The motivation is not well illustrated. I agree with that dropping sublayers could lead to training efficiency as the model turns to a shallower counterpart. However, I mean, pervious work like LayerDrop and others omit the layer computation in the forward pass. Then the computation could be removed in the subsequent backward computation with essential engineering efforts. Thus it lacks a clear distinction in terms of technical innovation compared to these previous works. - While the paper proposes omitting sublayer computation in the backward pass, it's unclear why the forward pass computation remains unchanged. Justifying this choice or exploring alternatives would strengthen the contribution. - The faster convergence observed in Figure 5 with DropBP compared to the vanilla model is counterintuitive. The observation here quite confuses me since the backward pass optimizes a partial computation graph, concerns regarding overfitting arise. The paper would benefit from a discussion on potential regularization techniques employed to address this, and a comparison with related work (e.g., [1]) that utilizes sublayer dropping for regularization in training a deep Transformer model. [1] Li et al., 2021 (AAAI) Learning Light-Weight Translation Models from Deep Transformer

Questions

some typos - Line 49: As a results -> As a result - Line 62: a effective -> an effective

Rating

6

Confidence

4

Soundness

3

Presentation

2

Contribution

2

Limitations

Yes

Summary

The paper proposes a novel method to reduce the computational and memory costs associated with fine-tuning large language models (LLMs). The authors introduce DropBP, a technique that randomly drops layers during backward propagation, effectively reducing the computational operations (FLOPs) and activation memory needed. This method assigns drop rates based on the sensitivity of each layer to ensure stable training. The approach is applicable to both full fine-tuning and parameter-efficient fine-tuning (PEFT) methods. The paper reports significant improvements in training time, convergence speed, and maximum sequence length when fine-tuning LLaMA2 models with DropBP.

Strengths

- DropBP introduces a novel method for reducing the computational and memory costs associated with fine-tuning LLMs. This is an important contribution to the field, given the increasing size and complexity of these models. - The paper provides empirical evidence that DropBP significantly reduces training time (by 44%), accelerates convergence (1.5× faster), and increases the maximum sequence length (up to 6.2×) on a single NVIDIA A100 GPU. These results demonstrate the effectiveness of the approach. The authors conduct thorough experiments on multiple datasets and models, providing a robust evaluation of DropBP's performance across different scenarios.

Weaknesses

- The paper mentions that the sensitivity calculation is done only once and has negligible overhead. However, more details on this process and its potential impact on training time would provide a clearer understanding of any trade-offs involved. - The paper could benefit from a more detailed theoretical analysis of why DropBP works as effectively as it does. This would strengthen the paper by providing a deeper understanding of the underlying principles.

Questions

- Can you provide more details on the sensitivity calculation process? Specifically, how is the sensitivity of each layer computed, and what is the computational overhead associated with this step? - What are the best practices for tuning the drop rates in DropBP? Are there guidelines or heuristics that practitioners can follow to optimize performance for their specific use cases? - How well does DropBP integrate with other recent advancements in efficient training techniques, such as mixed precision training or distributed training frameworks? Have you explored these combinations in your experiments?

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

Limitations have been discussed.

Summary

The paper proposed to drop layers during backward prop (BP) based on layer sensitivity. The method aims to reduce the cost for gradient computation and storage for intermediate activation in full BP.

Strengths

1. Reducing the cost of full BP in PEFT has been an important challenge. 2. The method is simple and is easy to integrate to either full fine-tuning or PEFT. 3. Experiments demonstrate that DropBP can speed up the training while retaining the accuracy. The resulting memory reduction makes longer sequence modeling accessible.

Weaknesses

1. The idea of optimizing NNs with sparse gradient is not new. This paper needs to add more discussion and comparison with related works in sparse learning e.g., [1-3] 2. Table 1 only shows results on two datasets and limited benchmark. It is unclear if the method works well for generation tasks and domain-specific transfer learning. 3. It is unclear which algorithm is used to solve the constraint minimization problem, i.e., to determine the layer-specific rates based on sensitivity, and its extra computational cost. 4. (Minor) In fine-tuning, DropBP drops a set of layers. However, the sensitivity of a set of layers may not be accurately represented by the direct summation of the sensitivities of individual layers in the set. [1] Sun, Xu, et al. "meprop: Sparsified back propagation for accelerated deep learning with reduced overfitting." [2] Sung, Yi-Lin, Varun Nair, and Colin A. Raffel. "Training neural networks with fixed sparse masks." [3] Brock, Andrew, et al. "Freezeout: Accelerate training by progressively freezing layers."

Questions

1. What is the long context modeling performance after applying DropBP? 2. Could the authors present Figure 5 with # of steps as the x-axis to demonstrate faster convergence? 3. I wonder if the sensitivities would evolve, and the drop rate needs to be re-allocated through training.

Rating

6

Confidence

3

Soundness

3

Presentation

3

Contribution

2

Limitations

The authors have addressed the limitations

Thanks for the response

Thanks the authors for providing new results and analysis in the updated version and I would like to keep my original score.

After reading the response

Thank you for the effort the authors have put into addressing my concerns. To this end, most of my concerns have been satisfactorily addressed. Specifically: - Regarding motivation, I agree that the authors have demonstrated that PLD-like models face convergence challenges during the training phase if the forward pass is dropped. While I still have some doubts about the results, more detailed discussion should be included in the next version to clarify this issue. It's not entirely clear to me whether these challenges are solely due to the increased model capacity, as larger models can sometimes be more robust to the training data than smaller ones. - The training curve plotted against training time, rather than training steps, seems meaningful. A better illustration of this would help others understand your work more clearly. Overall, I would like to slightly raise my score.