Dynamic Context Pruning for Efficient and Interpretable Autoregressive Transformers

Autoregressive Transformers adopted in Large Language Models (LLMs) are hard to scale to long sequences. Despite several works trying to reduce their computational cost, most of LLMs still adopt attention layers between all pairs of tokens in the sequence, thus incurring a quadratic cost. In this study, we present a novel approach that dynamically prunes contextual information while preserving the model's expressiveness, resulting in reduced memory and computational requirements during inference. Our method employs a learnable mechanism that determines which uninformative tokens can be dropped from the context at any point across the generation process. By doing so, our approach not only addresses performance concerns but also enhances interpretability, providing valuable insight into the model's decision-making process. Our technique can be applied to existing pre-trained models through a straightforward fine-tuning process, and the pruning strength can be specified by a sparsity parameter. Notably, our empirical findings demonstrate that we can effectively prune up to 80\% of the context without significant performance degradation on downstream tasks, offering a valuable tool for mitigating inference costs. Our reference implementation achieves up to $2\times$ increase in inference throughput and even greater memory savings.

Paper

References (66)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer jYJo7/10 · confidence 3/52023-07-03

Summary

This paper proposes an efficient and interpretable context-based dynamic pruning method. They use additional query / key layers to generate dynamic attention masks and sparsify self-attention maps. They also introduce a sparse sigmoid and regularization term to control the sparsity. In experiments, they demonstrate the lowest performance degradation compared to previous static attention pruning methods at the same level of sparsity. The results of throughput and speed analysis show that the proposed method can achieve additional inference efficiency with minimal performance loss. They analyze the distribution of remaining contexts with respect to part of speech and the depth of the layer. They also demonstrate that the proposed method dynamically prune attention based on contexts through context switch experiments.

Strengths

- The proposed method is well-motivated and easy to follow. - The proposed method efficiently increases interpretability and sparsity with minimal performance drop compared to previous research. - They provide an extensive analysis of the relationship between context length, throughput, and speed in various parameter sizes. - Based on the proposed pruning method, they devise efficient batched data structures for the optimized computation.

Weaknesses

- Quantitative comparison of throughput and speed with local/sparse attention would contribute to a comprehensive understanding. - Further qualitative study of interpretability (Fig. 8) varying the sparsity level (gamma) would provide additional intuitive observations.

Questions

Do you have expectations on how the zero-shot performance difference between the pruning strategies will change as the number of model parameters increases?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

As written in the Limitations section, scalability studies on larger language models (>7B) would provide further insights into the dynamic pruning method.

Reviewer uWWw7/10 · confidence 4/52023-07-03

Summary

The paper introduced a novel inference strategy for transformer models that focuses on inference efficiency. Instead of retaining all context tokens throughout the entire inference process, they gradually eliminate tokens as they move deeper into the layers. To determine which tokens to drop, they trained some small linear layers to predict the remaining relevant context. Their experiments revealed that approximately 80% of the tokens could be safely discarded without much adverse impact on downstream task performance or perplexity. As a result, this approach significantly reduces the computational resources required for inference when the context length exceeds 500 tokens.

Strengths

- The paper demonstrates excellent writing with a clear flow of ideas. - The authors introduce a unique and innovative data structure that enables batch operations involving masked tokens. - The experimental results indicate that in scenarios where long context (> 500 tokens) is involved in the inference process, the models can safely drop up to 80% of the tokens without any impact on perplexity.

Weaknesses

- **The choice of downstream tasks raises questions:** The downstream tasks evaluated in Figure 5 primarily involve small context sizes, and Figure 7 indicates that a smaller context leads to reduced throughput compared to the standard dense model. It would be more persuasive to demonstrate that task performance remains intact when longer contexts are required, while simultaneously achieving gains in inference efficiency. - **Insufficient experiments with stronger base models:** It appears that the proposed method inevitably leads to performance degradation for larger and more capable base models, as evident from both Figure 5 and 7. This is likely because stronger base models are better at utilizing contexts, and additional contextual information enhances performance. To strengthen the paper's argument, the authors should include more results using larger and stronger base models (>1.5B parameters), such as LLaMA, Pythia, or even OPT. - **Evaluating generation quality:** Though the authors perform evaluations on language modeling with perplexity, it does not necessarily align with generation quality. If would be helpful if the authors could further provide evidence that dropping context tokens does not affect generation quality.

Questions

- The paper introduces the use of sparse sigmoid to gradually enforce sparsity on the context tokens. It is important to understand why sparse sigmoid techniques were chosen and if they offer any specific advantages. Could other sparsity techniques, such as l0-regularization with hard-concrete distributions (Louizos et al., 2017), or techniques used in movement pruning (Sahn et al., 2020), achieve similar results? - There seems to be a typo in Figure 5. The green curve should be labeled as "Sparse Attention". - Figure 2 appears to be a bit confusing without proper legends. While it is clear that "X" denotes dropped tokens, the meaning of the red blocks is unclear. Could you provide an explanation or add appropriate legends?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

3 good

Limitations

The authors mentioned the limitation of the working being exclusively tested on autoregressive language models, and specifically GPT2 model family. The paper would be stronger if the author can show positive results on stronger base models.

Reviewer LhSf7/10 · confidence 4/52023-07-06

Summary

The authors propose a modified dynamic masking operation to the traditional multi-headed attention in transformers in order to allow models to learn to drop tokens at specific layers during training. In order to facilitate learning, they use a sparse sigmoid that is annealed to interpolate from a traditional sigmoid up to a step function in the infinite limit. They show that this dynamic sparse attention achieves lower perplexity at the same sparseness levels compared to other modified attention mechanisms, as well as better zero-shot accuracy and throughput. The proposed model shows promise for improving the efficiency and overcoming the quadratic time complexity of traditional dense transformers.

Strengths

Improving the efficiency of vanilla dense transformers is an important problem, and being able to learn the sparsification operation rather than setting a static prior is an interesting direction. The proposed method is simple and computationally efficient, and the manuscript is well written and clear. The results demonstrate marked improvements over other sparse attention mechanisms at similar sparsity levels as well as increased throughput.

Weaknesses

The main concerns I have are with how well this adaptive sparsity can be used when training from scratch and on long range dependency benchmarks such as Long Range Arena [1]. Since the experiments are initialized from GPT-2, all dense information is present, the model simply has to learn which information it can safely ignore. However, when training from scratch this optimization problem becomes much more difficult since falling into a local optimum with respect to pruning early on can reduce the model’s effectiveness in the future as the $\alpha$ annealing increases towards a step function. It would be interesting to see results (even on small models) on how much utilizing this mechanism during from scratch training affects perplexity, given the same compute budget. [1] Long Range Arena: A Benchmark for Efficient Transformers. 2020. Yi Tay, Mostafa Dehghani, Samira Abnar, Yikang Shen, Dara Bahri, Philip Pham, Jinfeng Rao, Liu Yang, Sebastian Ruder, Donald Metzler

Questions

- The $\alpha$-sigmoid operator is a bit confusing to me. How are argmax values calculated efficiently and how are gradients propagated through the argmax operator in equation 8? - Do the baseline GPT-2 models use FlashAttention in their implementation? If not, how does the throughput of the dynamically pruned model compare to one with FlashAttention? - How sensitive is the training to the correct annealing of the $\alpha$ parameter?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

3 good

Limitations

Yes

Reviewer 65eF7/10 · confidence 4/52023-07-07

Summary

Given the trend in large language models, it is a pretty important problem to search to efficient architectures. In this direction is the line of work to make the attention component efficient by introducing sparsity in the attention block and allowing every token to attend only a subset of the previous tokens. The paper presents usage of Adaptively Sparse Attention whereby 1) The network learns to drop parts of the context no longer required. 2) The tokens dropped at every layer are independent, and different set of tokens might be chosen to be persisted at every layer. 3) The network is trined using sparse sigmoid functions that introduce sparsity during training itself in contrast to some works that have tried to introduce sparsity only during inference. Experiments show that this methodology can lead to pretty strong performance (with minimal perplexity loss), even with 80% sparsity.

Strengths

The work presented in the paper is pretty innovative and impactful in this direction as finding efficient transformer architectures is key to sustain and grow the research and production usage of these large language models. The model successfully exploits the fact that the attention matrix in these models are pretty sparse and additionally encourages that with sparse sigmoid-like function to mask out some tokens. Experiments show pretty strong performance even with 80% sparsity in the network.

Weaknesses

Given that the model is performing pretty well in terms of perplexity even with huge sparsity, it would be interesting to perform an analysis of whether there is a class of NLP tasks the is significantly affected (like knowledge intensive tasks?), or perhaps translation or similar tasks where the structure of the input sentence matters if the model is focusing to forget stopwords very early.

Questions

- Was there any analysis done on which set of NLP tasks get affected the most with such sparsity? - For inference on a given device, do we have numbers on the increase in the max sequence length that can be supported? - Do these changes impact the training time throughput in any way?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

3 good

Contribution

3 good

Limitations

None discussed in paper, and nothing important that I can think of.

Reviewer 65eF2023-08-16

Thanks for the clarifications! Nice work. I have read through the responses and will keep my original scores.

Reviewer jYJo2023-08-16

Thank you for the clarification and additional results which are interesting. I have raised the rating after reading the rebuttal.

Reviewer LhSf2023-08-18

Thanks for the response. My questions have been answered but I will keep my score.

Reviewer uWWw2023-08-18

Thanks for your response!

I have read the response and think my questions and concerns are well explained and addressed. Though I highly recommend that the authors include larger models (LLaMA-7b) to strengthen the paper, I also believe the current results sufficiently demonstrate the key arguments. Thus I will keep the original score.

Program Chairsdecision2023-09-21

Decision

Accept (spotlight)

© 2026 NYSGPT2525 LLC