Mitigating Quantization Errors Due to Activation Spikes in GLU-Based LLMs

Modern large language models (LLMs) have established state-of-the-art performance through architectural improvements, but still require significant computational cost for inference. In an effort to reduce the inference cost, post-training quantization (PTQ) has become a popular approach, quantizing weights and activations to lower precision, such as INT8. In this paper, we reveal the challenges of activation quantization in GLU variants, which are widely used in feed-forward network (FFN) of modern LLMs, such as LLaMA family. The problem is that severe local quantization errors, caused by excessive magnitudes of activation in GLU variants, significantly degrade the performance of the quantized LLM. We denote these activations as activation spikes. Our further observations provide a systematic pattern of activation spikes: 1) The activation spikes occur in the FFN of specific layers, particularly in the early and late layers, 2) The activation spikes are dedicated to a couple of tokens, rather than being shared across a sequence. Based on our observations, we propose two empirical methods, Quantization-free Module (QFeM) and Quantization-free Prefix (QFeP), to isolate the activation spikes during quantization. Our extensive experiments validate the effectiveness of the proposed methods for the activation quantization, especially with coarse-grained scheme, of latest LLMs with GLU variants, including LLaMA-2/3, Mistral, Mixtral, SOLAR, and Gemma. In particular, our methods enhance the current alleviation techniques (e.g., SmoothQuant) that fail to control the activation spikes. Code is available at https://github.com/onnoo/activation-spikes.

Paper

Similar papers

Reviewer PpZ55/10 · confidence 4/52024-07-03

Summary

This paper pays attention to extremely large outliers in LLMs and further investigates the reasons behind these "attention spikes." Consequently, the authors propose two methods to enhance the performance of quantized models.

Strengths

1. The analysis of attention spikes is thorough and comprehensive. 2. The exploration of the relationship between attention spikes and Gated Linear Units (GLU) variants is both interesting and insightful.

Weaknesses

1. The proposed QFeM method is not hardware-friendly, as it maintains some modules at high precision and cannot directly utilize low-bit INT General Matrix Multiply (GEMM) for activations and weights. 2. The proposed QFeP method bears a strong resemblance to a previously researched method, IntactKV[1], yet lacks a detailed comparative discussion. 3. The experimental settings are limited to W8A8 configurations, which previous research, such as SmoothQuant[2], has shown can nearly achieve lossless quantization for W8A8 models. 4. The authors have not included comparisons with state-of-the-art baselines, such as OmniQuant[3], AffineQuant[4], QLLM[5], and QuaRot[6]. [1]. Liu, Ruikang, et al. "IntactKV: Improving Large Language Model Quantization by Keeping Pivot Tokens Intact." arXiv preprint arXiv:2403.01241 (2024). [2]. Xiao, Guangxuan, et al. "Smoothquant: Accurate and efficient post-training quantization for large language models." International Conference on Machine Learning. PMLR, 2023. [3]. Shao, Wenqi, et al. "Omniquant: Omnidirectionally calibrated quantization for large language models." arXiv preprint arXiv:2308.13137 (2023). [4]. Ma, Yuexiao, et al. "Affinequant: Affine transformation quantization for large language models." arXiv preprint arXiv:2403.12544 (2024). [5]. Liu, Jing, et al. "Qllm: Accurate and efficient low-bitwidth quantization for large language models." arXiv preprint arXiv:2310.08041 (2023). [6]. Ashkboos, Saleh, et al. "Quarot: Outlier-free 4-bit inference in rotated llms." arXiv preprint arXiv:2404.00456 (2024).

Questions

1. Could you provide a detailed analysis highlighting the differences between the QFeP method and IntactKV? 2. Could you expand the experimental results to include different quantization settings such as W4A4 and W4A8? 3. Could you offer a more detailed comparison with state-of-the-art (SOTA) baselines or conduct the ablation tests as outlined in Table 4? If the authors can provide more comprehensive results, I am prepared to raise my evaluation scores.

Rating

5

Confidence

4

Soundness

3

Presentation

2

Contribution

2

Limitations

Yes.

Reviewer PpZ52024-08-10

Minor questions about rebuttal

I appreciate the authors' discussion and the additional evaluation results provided. However, I have a few minor points that require further clarification. Firstly, IntactKV pre-saves all system prompts as pivot tokens for Vicuna models, which includes all attention spikes, similar to QFeP. Secondly, I am curious as to why QFeP does not improve the performance of the 4-bit LLaMA2-13B model in Appendix 3. Could this be due to the influence of an additional prefix affecting the model's performance?

Authorsrebuttal2024-08-12

We are sincerely grateful for your effort and time in reviewing our rebuttal and for your valuable questions. --- **Q5.** IntactKV pre-saves all system prompts as pivot tokens for Vicuna models, which includes all attention spikes, similar to QFeP. **A5.** The discussion continues from our previous response to your Q2, which highlighted the differences between QFeP and IntactKV [1]. Notably, IntactKV selects different pivot tokens depending on the type of LLMs. - For pre-trained LLMs (e.g., LLaMA), IntactKV selects only the [BOS] token as a pivot token. - For supervised fine-tuned LLMs (e.g., Vicuna), IntactKV stores all system prompts as pivot tokens. We denote these methods as IntactKV[B] and IntactKV[P], respectively, according to [1]. In our previous rebuttal, we compared QFeP with IntactKV[B] due to the absence of system prompts for pre-trained LLMs. When quantizing activations for pre-trained LLMs, QFeP's ability to address activation spikes more effectively than IntactKV becomes evident, as discussed in the previous response. Nevertheless, further investigation into activation spikes for supervised fine-tuned (or instruction fine-tuned) LLMs would be valuable. Previous works have found that pivot tokens persist after instruction fine-tuning [1, 2]. This finding implies that activation spikes are transferable and that our methods are also effective. Indeed, we observed equivalent activation spikes in both LLaMA-2 and its fine-tuned models, such as Vicuna-v1.5 and LLaMA-2-Chat. While IntactKV[P] may store long system prompts for pivot tokens, QFeP saves a more compact prefix with a length of 3. Furthermore, QFeP is explicit and prompt-agnostic, providing flexibility for system prompts. Finally, **QFeP can be applicable to more generalized and effective solutions for addressing activation spikes when applying activation quantization, regardless of whether the LLM is pre-trained or fine-tuned.** [1] Liu, Ruikang, et al. "IntactKV: Improving Large Language Model Quantization by Keeping Pivot Tokens Intact." arXiv preprint arXiv:2403.01241 (2024). [2] Sun, Mingjie, et al. "Massive activations in large language models." arXiv preprint arXiv:2402.17762 (2024). --- **Q6.** I am curious as to why QFeP does not improve the performance of the 4-bit LLaMA2-13B model in Appendix 3. Could this be due to the influence of an additional prefix affecting the model's performance? **A6.** During the preparation of the rebuttal to reviewer geCr, we confirmed that introducing only an additional prefix of QFeP slightly degrades the performance of the FP16 model. Based on this observation, we hypothesize the following regarding QFeP's contribution to the 4-bit LLaMA-2-13B: - Case 1 (Atom [3] + QFeP): Given that quantization errors have been minimized via the fine-grained group quantization scheme of Atom, the prepended prefix of QFeP has the possibility to slightly degrade performance. Note that the fine-grained group quantization utilizes a finer granularity than per-token quantization. - Case 2 (OmniQuant [4] + QFeP): At present, we are unable to determine the specific factor that degrades the perplexity for WikiText-2, although QFeP achieved a performance gain for C4. To provide further evaluation results, we assessed four zero-shot tasks, as shown in the table below. The results indicate that QFeP improves the OmniQuant baseline for three tasks, specifically the WinoGrande task. This is similar to the results of W4A6 LLaMA-2-13B in Table 2 of the attached PDF. While there are numerous potential factors that could explain the degradation (e.g., trainable components of OmniQuant or softmax quantization in low-bit), we are making an effort to include an ablation study, similar to Figure 6, to clarify the influence of an additional prefix. Thanks again for your questions. | Model | #Bits | Method | PIQA($\uparrow$) | LAMBADA($\uparrow$) | HellaSwag($\uparrow$) | WinoGrande($\uparrow$) | Avg($\uparrow$) | |---|---|---|:---:|:---:|:---:|:---:|:---:| |LLaMA-2-13B|FP16| - | 79.49% | 76.54% | 60.20% | 72.38% | 72.15% | |LLaMA-2-13B|W4A4|Atom|78.02%|75.80%|58.48%|70.56%|70.72%| |LLaMA-2-13B|W4A4|$\quad$+QFeM|78.07%|75.43%|**58.49%**|70.40%|70.60%| |LLaMA-2-13B|W4A4|$\quad$+QFeP|76.99%|**76.19%**|58.22%|70.80%|70.55%| |LLaMA-2-13B|W4A4|$\quad$+QFeM+QFeP|**78.24%**|75.61%|58.34%|**71.11%**|**70.83%**| |LLaMA-2-13B|W4A4|OmniQuant|67.25%|41.63%|44.95%|52.57%|51.60%| |LLaMA-2-13B|W4A4|$\quad$+QFeM|**71.11%**|**48.01%**|46.86%|57.14%|**55.78%**| |LLaMA-2-13B|W4A4|$\quad$+QFeP|68.93%|38.99%|45.95%|**58.48%**|53.09%| |LLaMA-2-13B|W4A4|$\quad$+QFeM+QFeP|70.02%|44.89%|**46.93%**|57.46%|54.82%| [3] Zhao, Yilong, et al. "Atom: Low-bit quantization for efficient and accurate llm serving." Proceedings of Machine Learning and Systems 6 (2024): 196-209. [4] Shao, Wenqi, et al. "Omniquant: Omnidirectionally calibrated quantization for large language models." arXiv preprint arXiv:2308.13137 (2023).

Reviewer TX3p5/10 · confidence 3/52024-07-08

Summary

This paper identifies some of the underlying causes for why activation quantization (PTQ) could lead to low performance and suggests some methods to address these issues.

Strengths

Please see the “Questions” section.

Weaknesses

Please see the “Questions” section.

Questions

My review is as follows: 1) The results of Table 2 seem to suggest that SmoothQuant leads to an unacceptably high performance degradation. Table 6 of the SmoothQuant paper however shows that for Llamav1, the degradation is very small. I wonder if a difference in implementation is causing this discrepancy in results. I’m sorry if this is mentioned somewhere in the text and I missed it. Could you please clarify? 2) What is the latency for FP16 in Figure 7? 3) There are multiple recent works that suggest quantizing the weights to 4 bits can be done without a big impact to the accuracy. Since memory access (for weights) is typically the bottleneck when LLMs are deployed, it is possible that one may prefer W4A16 over W8A8. In my opinion, studying 4-bit weight quantization (in addition to the already studied W8 quantization) could make the paper more interesting. 4) QFeM method is essentially a mixed precision approach. A lot of quantization papers actually employ some form of mixed precision (even though it is sometimes only mentioned in the footnote). For instance, SmoothQuant uses FP16 for LayerNorm. Has this been taken into account when comparing against other methods? Also, have similar assumptions been made in the implementation of the proposed method? Minor: 5) There is a type on line 90: “in the remain(der) of this 91 paper” 6) Typo in line 147 “dominate(s)” Readability and presentation: The paper is mostly easy to understand. One thing I could say is that the idea of QFeM is easier to follow than QFeP; the introduction of the QFeP method may need to be improved and made more concise.

Rating

5

Confidence

3

Soundness

2

Presentation

3

Contribution

3

Limitations

Yes.

Reviewer geCr5/10 · confidence 3/52024-07-10

Summary

This paper addresses the precision challenges posed by the large language models (LLMs) quantization during inference, specifically focusing on the quantization errors in GLU-based feedforward networks. The authors identify that GLU variants in LLMs cause significant local quantization errors due to excessive activation magnitudes, referred to as activation spikes. They observe that GLU-implemented models have larger spikes than non-GLU-implemented models. They propose two methods, Quantization-free Module (QFeM) and Quantization-free Prefix (QFeP), to isolate and mitigate these spikes during quantization. QFeM leave some linear layers unquantized (usually those layers that cause large activation spikes in the first several layers), and QFeP introduce an additional prefix before the inference process. Their extensive experiments show that these methods improve quantization performance and are compatible with existing techniques.

Strengths

1. The identification of activation spikes in GLU-based LLMs is novel. 2. The paper is well-structured and clear. 3. The QFeP method is novel, and is somehow similar to the finding of "sink token" in StreamLLM [1]. [1] Xiao, Guangxuan, et al. "Efficient streaming language models with attention sinks." arXiv preprint arXiv:2309.17453 (2023).

Weaknesses

1. My major concern is about the baseline of SmoothQuant reported in Table 4. For example, In Table 7 of SmoothQuant's original paper, they report that W8A8 SQ's PPL of Llama-7B on WikiText-2 dataset is 5.515, while the authors report a PPL of 9.907 on the same dataset. Is there a specific reason about this large gap? 2. In Table 3, the improvement brought by the QFeP method does not seem significant, especially when combining with the QFeM method.

Questions

1. In the method of Quantization-free Prefix (QFeP), will the additional prefix decrease or increase the accuracy? That means you use the QFeP method but do not apply quantization, only introduce the additional prefix. 2. Where do the spikes of the GLU-based model come from in the element-wise multiplication? Do they mostly comes from the gate projection part, or the up projection part?

Rating

5

Confidence

3

Soundness

3

Presentation

3

Contribution

2

Limitations

None

Reviewer BgAS5/10 · confidence 2/52024-07-11

Summary

This paper introduces activation quantization methods for GLU-based LLMs, which often face challenges due to activation spikes. To effectively manage these spikes and enable activation quantization using a PTQ-based approach, the paper proposes a Quantization-free Module (QFeM) and a Quantization-free Prefix (QFeP). Specifically, QFeM aims to partially bypass quantization for linear layers where large quantization errors occur. QFeP identifies the prefix that triggers activation spikes and preserves its context as a key-value (KV) cache, preventing the recurrence of activation spikes in subsequent tokens. The paper presents extensive experimental results to compare the accuracy of the quantized models.

Strengths

1) This paper is well organized and easy to understand. 2) The proposed QFeM and QFeP effectively mitigate the impact of activation spikes on activation quantization, preserving the accuracy of LLMs even when activation quantization is applied. 3) The ablation study thoroughly examines the effects of QFeM and QFeP, providing valuable insights.

Weaknesses

1) The perplexity/accuracy results of the baseline methods deviate from the results reported in previous papers. 2) The paper does not compare its method with the state-of-the-art LLM quantization method [1], which enables W4A4 quantization (partially using 8-bit operations) with a PTQ approach. [1] Zhao, Yilong, et al. "Atom: Low-bit quantization for efficient and accurate llm serving." Proceedings of Machine Learning and Systems 6 (2024): 196-209.

Questions

1) The perplexity and accuracy results of the baseline methods (SQ [1] and OSP [2]) in Table 4 are worse than the figures reported in the original SQ and OSP papers. For instance, the SQ paper reported successful preservation of LLM perplexity after W8A8 quantization (Table 7 of [1]), but Table 4 of this paper shows poor perplexity and accuracy results for SQ. Additionally, the OSP paper claimed successful preservation of LLM perplexity even after INT6 quantization and reported better perplexity results compared to SQ (Table 2 of [2]). Although OSP only evaluated LLaMA-1 and there is no LLaMA-2 data, we can reasonably expect similar trends for LLaMA-2 given that both use GLU-based activation functions. However, Table 4 of this paper shows poor perplexity and accuracy results for OSP, particularly for LLaMA-2-7B. Why are the evaluation results for the previous methods so different? 2) Since the prefix token is retrieved from the calibration set and the threshold alpha is determined from it, will these parameters remain consistent when inferring on a new dataset using the same model? 3) What are the advantages of the proposed method compared to Atom [3] in terms of perplexity/accuracy, latency, or other aspects? [1] Xiao, Guangxuan, et al. "Smoothquant: Accurate and efficient post-training quantization for large language models." International Conference on Machine Learning. 2023. [2] Wei, Xiuying, et al. "Outlier suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling." arXiv preprint arXiv:2304.09145 (2023). [3] Zhao, Yilong, et al. "Atom: Low-bit quantization for efficient and accurate llm serving." Proceedings of Machine Learning and Systems 6 (2024): 196-209.

Rating

5

Confidence

2

Soundness

2

Presentation

2

Contribution

2

Limitations

The proposed method is limited to GLU-based LLMs.

Reviewer geCr2024-08-08

The results are very interesting. I increase my score accordingly.

Reviewer TX3p2024-08-12

Thank you for the detailed responses! It would be great the findings reported in the rebuttal could be explicitly incorporated in the revision. I'll increase my rating from 4 to 5 since some of my concerns regarding the numerical results are already addressed.

Authorsrebuttal2024-08-12

Thank you for your insightful reviews of our work and rebuttal during the review period. Your valuable feedback is greatly appreciated and will be incorporated into the final version.

Reviewer BgAS2024-08-13

Thank you for your response and the additional experiments. You have addressed many of the questions I had, so I have decided to increase my score.

Authorsrebuttal2024-08-13

We greatly appreciate your time and effort in reviewing our rebuttal. Your constructive questions have been instrumental in enhancing the quality of our work.

Program Chairsdecision2024-09-25

Decision

Reject

© 2026 NYSGPT2525 LLC