Exploiting LLM Quantization

Summary

This paper exploits the discrepancy between the full-precision and the quantized model to initiate attacks. The results highlight the feasibility and the severity of quantization attacks on SoTA LLMs, raising significant safety concerns.

Strengths

* This paper is well-written with clear motivation and illustrations. * The potential problem studied in this paper is impactful for real-world applications. * The topic of safety concerns in LLM quantization is novel.

Weaknesses

* The title might be too broad, it would be better to add more specific terms. * It would be better to include some larger widely employed LLMs in evaluations, for example, the Mixtral-8x7B model. * Including different quantization methods will make this paper more solid, for example, GPTQ and AWQ.

Questions

* The models evaluated in this paper are pra-trained LLMs, I’m interested in the performance of aligned LLMs. Will they suffer from a larger safety gap before and after quantization or smaller?

Rating

7

Confidence

4

Soundness

2

Presentation

2

Contribution

3

Limitations

Yes.

Summary

The paper points to a potential vulnerability that an attack modifies a full-precision LLM that the full-precision LLM behaves well but after quantizing the model, it can have some harmful responses. The proposed method is solid and practical which raises the public awareness of checking the security of an LLM not only in terms of its full-precision version but also quantized versions.

Strengths

1. The paper studies a very interesting question that how can we construct a well-behaved LLM in full precision but can be malicious after being quantized. The authors showed this is possible through popular quantization methods. 2. The paper raises people's awareness that besides checking the full-precision model, we also need to check their quantized version before we release the model.

Weaknesses

1. The threat model is not well-explained. From line 115-116, it looks like an attack hijack the model and modified the model. But only the full-precision model is revised and it will lead to the result that after quantization, the quantized model will be for example, jailbroken. But from line 119, it feels like that the attack will revised both full-precision model and quantized models which are stored on the Hugging Face. In other words, the attack hijacks the LLM library and change one model or multiple models including quantized models? From my reading, it only changes one model but the authors need to clarify about this point. 2. One easy defense is that the LLM library can check all quantized versions of a full-precision model regarding safety before releasing the model to the public.

Questions

1. In experimental details, what are the clear definition of your metrics? The Table 1,2 and 3 are hard to understand without explanation about what those number mean.

Rating

7

Confidence

4

Soundness

3

Presentation

2

Contribution

4

Limitations

Discussed in the last section.

Summary

This paper studies the idea of exploiting quantization as an attack vector. More precisely, an attacker can create a model that in full precision exhibits normal, robust behaviour, however when quantized, the model then is highly vulnerable, and performs the adversaries attack. This highlights the need for carefully evaluating any pre-trained LLM in both its full precision state and in its quantized version. Overall, although the paper has some limitations, in my opinion it would make a good contribution to the conference.

Strengths

The paper shows a very relevant and pertinent attack vector: downloading LLMs through Huggingface is one of the de-facto ways people obtain pre-trained models, and so a stealthy attack such as this one is of significant real-world importance. The paper has been evaluated on several different tasks (code generation, over-refusal, and content injection) demonstrating the general applicability across domains and attack objectives. The paper points to some interesting further observations and results, with both analysis of how the spread of LLM's weights influence the attack, and a potential defence in the form of Gaussian noise injections. The results are preliminary (e.g. what if the attacker knows that Gaussian noise is being added, would they be able to adapt their attack?), but point to further work in this area.

Weaknesses

It seems like a baseline that is missing in many of the Tables would be the performance of the clean, original LLM, when quantized. Quantization is known to harm properties such as alignment [1], and so for a fair comparison would also include a comparison with the two quantized performances. The attack does have some novelty limitations, as it is based on an existing attack (which the authors are very clear and upfront about). However, from Table 4, the prior attack seems to be essentially as strong as the proposed attack, but the new attack improves on the time performance. Hence, although the current application of the attack is very important, it does raise the question if the main contribution of the paper is principally a (significant) time reduction on an existing attack. [1] Kumar, Divyanshu, et al. "Increased llm vulnerabilities from fine-tuning and quantization." arXiv preprint arXiv:2404.04392 (2024).

Questions

1) It is unclear to me how step (3) of the attack is carried out: how can the attacker make sure that their optimizations on the malicious model push it to be a benign model? Does the attacker carry out full safety training themselves locally? 2) It was somewhat unclear at times the full delta between this attack and the original Ma et al are. Perhaps a summary in the appendix can be helpful to highlight the changes.

Rating

7

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

Limitations have been adequately addressed, e.g. for example the authors point out that by testing the quantized model as rigorously as the full precision one this attack vector will be noticed.

Summary

The paper investigates the security vulnerabilities introduced by quantizing LLMs to lower-precision weights, a common technique used to reduce memory usage and facilitate deployment on commodity hardware. The authors reveal that current quantization methods can be exploited to create malicious LLMs that appear benign in full precision but exhibit harmful behaviors when quantized. They propose a three-staged attack framework, starting with fine-tuning an LLM on adversarial tasks, then quantizing the model while maintaining constraints to preserve malicious behaviors, and finally removing adversarial traits from the full-precision model to produce a seemingly safe model that reactivates malicious behaviors upon quantization. Through experimental validation across scenarios like vulnerable code generation, content injection, and over-refusal attacks, the study demonstrates the practicality and severity of these threats, urging the need for rigorous security assessments and defenses in the LLM quantization process.

Strengths

1) The paper is well written and well organized. 2) The authors have tried to address a pertinent concern associated with security of LLMs.

Weaknesses

1) While the paper claims to be the first to study the security implications of LLM quantization, it builds on well-established concepts such as model quantization and adversarial attacks. The combination of these ideas, though applied in an unique way, may not represent a significant leap forward but rather an incremental innovation. 2) The techniques used in the paper, such as fine-tuning on adversarial tasks and using projected gradient descent (PGD), are well-known in the literature. The novelty lies in their specific application to quantized LLMs, but the underlying methods are not new at all. 3) The complexity and practicality of the attack in real-world scenarios could be further scrutinized. Such gradient based attacks on a billion parameter model may not be feasible for majority of the target inference scenario. So, the authors need to motivate the attack scenarios better. Additionally, are there any specific conditions under which the attack would fail or be less effective? 4) The evaluation focuses on specific quantization methods (e.g., LLM.int8(), NF4, FP4). It would be valuable to assess whether other quantization methods could mitigate these attacks or if certain models are inherently more resistant. 5) The experiments demonstrate the feasibility of the attack across different scenarios. However, the robustness of these experiments could be improved: • The diversity of the datasets and models used for evaluation could be expanded. • The paper could include more comprehensive analysis along with comparison with existing research to strengthen the claims. 6) What happens to models optimized via methods like KV cache quantization [1-2], or weight reparameterization [3]? [1] GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM, arxiv 2024 [2] KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache, ICML 2024 [3] ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterization, arxiv 2024

Questions

Please refer to weakness.

Rating

5

Confidence

4

Soundness

2

Presentation

2

Contribution

2

Limitations

The contribution is built on top of well established ideas, and the demonstration and evaluations are limited.

Response to the Rebuttal

Thanks for submitting the rebuttal. I have read it and will keep the score.