Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing

Transformer models have been widely adopted in various domains over the last years, and especially large language models have advanced the field of AI significantly. Due to their size, the capability of these networks has increased tremendously, but this has come at the cost of a significant increase in necessary compute. Quantization is one of the most effective ways to reduce the computational time and memory consumption of neural networks. Many studies have shown, however, that modern transformer models tend to learn strong outliers in their activations, making them difficult to quantize. To retain acceptable performance, the existence of these outliers requires activations to be in higher bitwidth or the use of different numeric formats, extra fine-tuning, or other workarounds. We show that strong outliers are related to very specific behavior of attention heads that try to learn a "no-op" or just a partial update of the residual. To achieve the exact zeros needed in the attention matrix for a no-update, the input to the softmax is pushed to be larger and larger during training, causing outliers in other parts of the network. Based on these observations, we propose two simple (independent) modifications to the attention mechanism - clipped softmax and gated attention. We empirically show that models pre-trained using our methods learn significantly smaller outliers while maintaining and sometimes even improving the floating-point task performance. This enables us to quantize transformers to full INT8 quantization of the activations without any additional effort. We demonstrate the effectiveness of our methods on both language models (BERT, OPT) and vision transformers.

Paper

Similar papers

Peer review

Reviewer Akrm7/10 · confidence 5/52023-06-26

Summary

This paper has thoroughly analyzed the activation outlier problem during quantizing of the transformer model, and further indicates the limitation of existing works. Based on the comprehensive studies, the author proposes two components, namely clipped softmax and gated attention, for regularizing the activation outlier during training. Sufficient experiment results demonstrate the effectiveness and efficiency of the proposed method.

Strengths

$\cdot$ The proposed components, namely clipped softmax and gated attention are simple but reasonable. Further experiments also support their effectiveness. $\cdot$ The observation is solid and further analysis is practical. The experiments include different structures of the transformer model in various tasks. $\cdot$ I appreciate the comprehensive studies conducted by the authors. The experiment result of computing cost is convincing. $\cdot$ This paper is well-organized and clear to understand. And The authors provide detailed information on related methods, making it easy for readers to understand the progress of the field.

Weaknesses

$\cdot$ The quantization performance of the proposed method at a lower bit (e.g. 4W4A) is worth looking forward to.

Questions

$\cdot$ The avg. kurtosis of OPT model with clipped softmax in tab. 2 is strange, what is the possible reason?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

4 excellent

Presentation

3 good

Contribution

3 good

Limitations

$\cdot$ The author has claimed in L290-296. I encourage the authors to finish the experiment in the larger model. $\cdot$ I encourage the authors to release the code for reproducing.

Reviewer NwNc6/10 · confidence 3/52023-07-03

Summary

The paper proposes two architectural modifications that mitigate the activation outlier problem that makes transformers challenging to quantize. Particularly, the authors conduct a comprehensive analysis of the underlying causes of outlier issues in various pre-trained Transformer models across different tasks and datasets. They identify specific architectural limitations that lead to large outliers during the pre-training phase. To tackle this challenge, the authors propose two solutions: 1/ clipped softmax and 2/ gated attention that can effectively address the outlier problem without sacrificing model performance when compared to vanilla Transformer models across various use cases. Furthermore, the suggested modifications result in smaller outlier values and enhance the quantization performance of the Transformer models, making them more robust to quantization.

Strengths

1. The paper presents strong motivations and novel approaches to address outlier issues that distinguish it from prior works. Unlike existing methods that focus on circumventing the impact through post-training techniques, this paper proposes a fundamental training-time solution. 2. The paper proposes a simple yet highly effective methodology, which directly addresses the fundamental issues and observations.

Weaknesses

1. Evaluating the proposed methodology by comparing it solely to the naive application of (vanilla) PTQ on the plain Transformer architecture might be somewhat unfair and not informative enough. Various outlier suppression methods have been proposed [] as post-training solutions, and thus it would be better to include a comparison with these methods. Given that the proposed methodology requires from-scratch training, its practical value will be justified if it outperforms post-training methods that require no additional training. 2. The paper introduces two distinct solutions; however, they are two orthogonal/competing methodologies that tackle the same problem. The paper lacks clarity regarding the specific circumstances under which each solution should be employed. That said, how can we decide which scheme to use when training a new model on a new dataset without having to try both and choose the better one? 3. As also stated in the paper, the evaluations have only been conducted on smaller-scale models (100M parameters). While the proposed architectural and train-time modifications show promising results that match/outperform the vanilla Transformer performance, it is unclear if this trend can persist on a larger scale (hundreds of millions to billions of parameters). It would be more informative if the authors can provide across different model scale regimes. Moreover, since the proposed solution is a pre-training methodology, it requires training from scratch and additional risk/cost related to extra hyperparameter tuning, which can potentially limit their practical application. [1] Wei, Xiuying, et al. "Outlier suppression: Pushing the limit of low-bit transformer language models." Advances in Neural Information Processing Systems 35 (2022): 17402-17414. [2] Xiao, Guangxuan, et al. "Smoothquant: Accurate and efficient post-training quantization for large language models." arXiv preprint arXiv:2211.10438 (2022).

Questions

N/A

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

Please see the weakness section

Reviewer Vxgc6/10 · confidence 4/52023-07-04

Summary

In this paper, the authors show that strong outliers are related to very specific behavior of attention heads that try to learn a “no-op” or just a partial update of the residual. To reduce outliers, the authors propose two simple (independent) modifications to the attention mechanism. This enables them to quantize transformers to full INT8 quantization of the activations without any additional effort.

Strengths

1. The authors provide a detailed analysis of the outliers in transformers, which well clarifies their research motivation and provides insights. 2. The analysis of "strong outliers are related to attention heads try to learn no-op" is interesting.

Weaknesses

1. The authors do not give detailed data distribution after training with the proposed method. I would expect a reduction in the magnitude and number of outliers than in the performance of INT8 quantization that validates the main target of the proposed technique. 2. The authors mention that it is limited to small models only (125M, which is too small), but quantizing small models with less than 2.7B parameters to INT8 is less challenging compared with large models [1]. There exists concern is the generalization ability of the proposed method on large-scale models. Analysis or experiments that demonstrate the effectiveness of proposed technique on larger scale model is recommended. [1] LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale. NIPS'22

Questions

1. Could the proposed method be applied to lower bitwidths beyond INT8? (e.g., 4/6 bit) 2. Does the proposed method require a full retraining of the FP model? When applied to larger models, the training cost could be excessive.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The authors do not include the limitations and potential negative societal impact in their paper.

Reviewer sWJn6/10 · confidence 5/52023-07-07

Summary

This paper analyzes the activation outlier problem. It is shown that the outliers are related to the behavior of transformer networks trying to learn not to update residuals (no-op). To achieve the exact zeros needed in the attention matrix for a no-update, the input to the softmax is pushed to be larger and larger. To solve this problem, the authors propose clipped softmax method and gated attention method, which can introduce exact zeros for softmax, thus the outliers can be removed.

Strengths

The paper explains why outliers exist in transformers. The analysis and visualization are convincing. Two independent methods are proposed (clipped softmax and gated attention) to solve the outlier problem of transformers, so that models can be quantized easily. Experiments on language models (BERT, OPT) and vision transformers are provided.

Weaknesses

No results on large scale models and datasets are provided.

Questions

How to explain the clipped softmax method no better than vanilla network on OPT? Can you provide the visualization (Figure 2) after using the proposed methods?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

Yes

Reviewer sWJn2023-08-17

Response to rebuttal

Thanks for the reply. Figure 1 in the attached PDF uses different head and sequence for clipped softmax and vanilla softmax. I suggest to use the same setting for visualization, even if the visualization results are not as expected, it will help understanding the method.

Authorsrebuttal2023-08-18

Thanks for the comment. Initially, we looked at and analyzed the same head and same sequence but decided to showcase a different head and a sequence with a similar attention pattern of soft no-update/partial update (we would show both if not the 1-page limit for the attached PDF). Such a pattern was not as pronounced as was the case presented in Figure 2 in the main text, but it's also not surprising (because of different random initialization & training dynamics it doesn't have to be the same exact head). However, we agree that including the same head & sequence is also valuable and we include both and possibly a few more for the camera-ready version.

Reviewer Vxgc2023-08-18

Thanks for the authors' response

Thanks for the authors' response. I keep my rating that recommends accept.

Reviewer NwNc2023-08-20

I appreciate the authors for their thoughtful response. I will keep my rating and stand for acceptance.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC