On the Exploitability of Instruction Tuning

Instruction tuning is an effective technique to align large language models (LLMs) with human intents. In this work, we investigate how an adversary can exploit instruction tuning by injecting specific instruction-following examples into the training data that intentionally changes the model's behavior. For example, an adversary can achieve content injection by injecting training examples that mention target content and eliciting such behavior from downstream models. To achieve this goal, we propose \textit{AutoPoison}, an automated data poisoning pipeline. It naturally and coherently incorporates versatile attack goals into poisoned data with the help of an oracle LLM. We showcase two example attacks: content injection and over-refusal attacks, each aiming to induce a specific exploitable behavior. We quantify and benchmark the strength and the stealthiness of our data poisoning scheme. Our results show that AutoPoison allows an adversary to change a model's behavior by poisoning only a small fraction of data while maintaining a high level of stealthiness in the poisoned examples. We hope our work sheds light on how data quality affects the behavior of instruction-tuned models and raises awareness of the importance of data quality for responsible deployments of LLMs. Code is available at \url{https://github.com/azshue/AutoPoison}.

Paper

References (56)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer oPzU6/10 · confidence 5/52023-06-25

Summary

The authors propose AutoPoison, an automated data poisoning pipeline. They demonstrate two types of attacks: content injection (e.g, brand names), and over-refusal attacks. The authors demonstrate how AutoPoison can change a model’s behavior by poisoning only a small fraction of data while maintaining a high level of stealthiness in the poisoned examples.

Strengths

1) The paper is well-written and easy to follow. 2) The idea is simple and intuitive, yet new at least in terms of the proposed attack pipeline.

Weaknesses

1) I'm mainly missing comparisons on more recent models, such as MPT, Falcon, LLaMA, etc. 2) the evaluation protocol for the proposed over-refusal attack should be much broader in order to extract meaningful insights from it. 3) Can the authors succeed on open source oracles? without using OpenAI's GPT? I now that some of the questions were mentioned in the limitations section, but in my opinion they are critical question that needs to be addressed.

Questions

See the Weaknesses section.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

authors addressed the limitations and, if applicable, potential negative societal impact of their work.

Reviewer 9ytD6/10 · confidence 4/52023-07-05

Summary

This paper analyzes the threat model for poisoning the data for instruction tuning to steer a language model towards certain behaviors. It proposes a data poisoning pipeline AutoPoison, that uses an oracle LLM to generate adversarial clean labels from perturbed adversarial instructions. AutoPoison is evaluated on two instances - content injection and over refusal - and outperforms the hand-craft baseline in terms of effectiveness and stealthiness.

Strengths

1. Creating a threat model for open-ended text generation and proposing a pipeline for it are novel and necessary in the era of LLMs. Focusing on instruction tuning rather than pre-training is reasonable due to the significantly smaller amount of data required for instruction tuning compared to model pretraining. 2. The analysis of model sizes and data poison ratios is thorough and provides people with a clear idea of how exploitable differently-sized models are under various settings. 3. Although the primary experiments utilize simplistic prompts for two rather basic settings of data poisoning, Section 5 demonstrates the extensibility and potential for the pipeline to be adapted for more specific and fine-grained use cases.

Weaknesses

My concerns about the paper is mostly about the evaluation. ### 1. The evaluation on stealthiness is limited. The paper's argument for stealthiness is that if the generation quality of a poisoned model is similar to that of a clean model, the poisoning pipeline is stealthy enough. This argument is reasonable to me. However, **I'm not sure perplexity and coherence score is good enough to measure the text quality** under the instruction tuning setting. While they can certainly measure the surface-level fluency of the language, the quality of an instruction-tuned model is more about whether it can faithfully follow the instruction. I think LLM-based metrics might be better in evaluating instruction following ability (such as what they did in Vicuna). I would be more convinced if you used human evaluation or LLM-based evaluation to justify the text quality. ### 2. Other aspects of the model should be considered, for example faithfulness and factuality. It seems to me that the proposed content injection attack may lead to more hallucinations that contain factual errors. For example, in the third example in Figure 3, the model lists McDonald's as a Swedish company, which is not true. I think these factual errors and other issues caused by hallucination cannot be measured by perplexity and coherence, but they are important for the model's performance. Therefore it may be better to consider a more comprehensive evaluation of the ability of the model. ### 3. Lack of human evaluation While the examples given look coherent and follow the instruction in a correct way. I can still tell the attacked model from the clean model by spotting the fact that many outputs contain "McDonald's" in a weird way. Although injecting this phrase is the goal of the attack, making it easy for humans to tell can easily leak the adversary's intention. It would be better if you conduct some sort of human evaluation to check if an ordinary user without prior knowledge of the attack can tell an attacked model from a clean one. Another concern is the limited discussion about defense strategies. I think it would be better to have some preliminary discussion on how the proposed pipeline might be defended.

Questions

1. What are some practical use cases for the attack? 2. Do you think the attack can be stealthy enough so that real humans cannot tell? 3. Considering the fact that instruction datasets are much smaller and often annotated by human beings, do you think the threat model is reasonable? How can an adversary attack the annotation process?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

3 good

Contribution

3 good

Limitations

The authors have addressed the limitations.

Reviewer AQ2w7/10 · confidence 4/52023-07-06

Summary

This paper proposes AutoPoison, an approach that automatically constructs poisoning data for instruction tuning. AutoPoison replaces training responses with poisoned responses obtained by querying an oracle LM with poisoned instructions. AutoPoison is evaluated on two tasks, content injection and over-refusal attacks. The experimental results show that AutoPoison achieves effective attack while maintaining overall text quality.

Strengths

The paper has the following strengths: 1. The paper is well-written and easy-to-follow. It includes sufficient examples helpful for understanding. 2. The approach is simple (in a good way) and can be generalizable. 3. The evaluation is thorough and clearly demonstrates AutoPoison’s effectiveness.

Weaknesses

### The capabilities of instruction-tuned LMs A key question that the paper did not answer is if the data poisoning significantly deteriorates the LM’s capabilities. I am aware that text generated by the poisoned model has low perplexity and is coherent with the instruction. But these two metrics do not fully represent the LM’s capabilities. I would suggest evaluating the LM’s capabilities on any standard benchmark, such as HELM and MMLU. If the poisoning does not significantly deteriorate the capabilities, that strengthens the paper’s contribution. Otherwise, the attack is not stealthy, because once the users find out that the LM cannot do some task, they will stop using the LM. ### Other small issues 1. Which oracle model did you use in your experiments? Line 136 only says that the oracle model can be GPT-3.5-turbo. 2. At Line 195, why did you use greedy decoding? Typically text is sampled from LMs, e.g., ChatGPT. 3. For the third example in Table 4, McDonald’s does not have a bold font.

Questions

Please consider addressing the points raised in the “Weakness” section.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

I believe that the paper provides a sufficient discussion of limitations and potential negative impact.

Reviewer uhYd7/10 · confidence 4/52023-07-09

Summary

This paper proposed AutoPoison, an automated data poisoning pipeline to showcase two example attacks: content injection and over-refusal attacks over the instruction-tuned models. Overall, this is a nice paper, I appreciate the authors for tackling this research problem. Authors proposed a sound methodology to model two types of attack. My main criticism is at evaluating the attack success, mainly at the metrics that have chosen to report. I would suggest authors to revisit this section to ground this work.

Strengths

* Timely work that shows eliciting exploitable behaviors from downstream models by injecting poisoned instructions. * Demonstrate two example attacks with different target behaviors: content injection and over-refusal attacks * Show that the success rates of the content injection attacks correlate with the scale of the LLM. It actually signifies with the machine-generated data. > “Intriguingly, we find that larger models, empowered with stronger language modeling and generalization ability, are more susceptible to content injection” * Nice metric to evaluate the over-refusal attacks via two -staged informative refusal with responses and reasons.

Weaknesses

* Author used machine-generated data for instruction tuning. Given that they use GPT-3.5-turbo as the oracle model, the poisoned responses may also seem to be comparable in fluency. Thus, I don’t think perplexity is a right metric to evaluate the attack's stealthiness. > “For instruction tuning, we use the English split of GPT-4-LLM, an open-source dataset of machine-generated instruction-following data.” * It seems like the performance advantage of the hand-crafted baseline might be due to the coherence score calculation. Need better clarification here. * This is strange. Need explanations. > “In addition, we observe that under the over-refusal attack, OPT-1.3B, the middle-sized model, learns this behavior the fastest.”

Questions

* Threat model considers the responses to be semantically meaningful and to achieve a qualitative change in model behavior. Are they not two competing objectives? * Is the over-refusal attack a specialized version of the content injection attack? > “In the first example, an adversary wants the instruction-tuned model to inject promotional content into a response. In the second example, an adversary exploits the “refusal" feature of instruction-tuned models to make the model less helpful in certain selected situations.” * What kind of biases inherent to oracle models when generating poison instructions? * Does this assume that the fluent responses will make it easier in the LLM finetuning process? What about over-fitting the LLM to such examples? > “Because r_adv is crafted by a language model and not a human, this automated response will already have low entropy according to the language model, making it easy to elevate the likelihood of this response during fine-tuning without a severe change in behavior.” * Authors argue that poisoned responses are hard to detect manually. Did authors perform any qualitative experiments to support this claim? > “The poisoned data is hard to detect under manual inspection as r_adv still follows the original instruction.” * Is the coherence score calculated between the generated response and the gold standard response? I am confused with the following statement. > “We measure the coherence between the instruction and the model’s response in our setting.”

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

Authors adequately addressed the limitations

Reviewer AQ2w2023-08-14

I have read other reviews and the author rebuttals. I would like to thank the authors for providing the new experiment results, which are convincing. Therefore, I raise my rating from 6 to 7.

Reviewer oPzU2023-08-19

I thank the authors for their response. I appreciate the evaluation of AutoPoison with an open-source oracle, but without the evaluation of more recent models, I will keep my initial positive score. Wish you good luck with the final decision!

Reviewer uhYd2023-08-20

Thank you for authors addressing my comments on the evaluation. I appreciate the authors who took time for additional experiments, and verified some claims presented in the original paper. Hereby, I raised my original score from 5 to 7 since I am satisfied with the rebuttal. Few additional verifications; * Can authors also present the prompt template used in the LLM evaluation. > We report two sets of scores using GPT-4 and GPT-3.5-turbo as judges

Authorsrebuttal2023-08-21

Thank you for acknowledging our rebuttal and updating the recommendation. For the LLM-based evaluation, we use the default prompts provided by MT-Bench [1], which are in their official repository (under the path `fastchat/llm_judge/data/judge_prompts.jsonl`). MT-Bench differs from the original vicuna evaluation in that it uses score-based single-answer gradings, so the prompts we use are those labeled as `"type": single`. We will include the evaluation details in our updated appendix. [1]. Zheng et al. “Judging LLM-as-a-judge with MT-Bench and Chatbot Arena.” arXiv 2023.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC