Protecting Your LLMs with Information Bottleneck

The advent of large language models (LLMs) has revolutionized the field of natural language processing, yet they might be attacked to produce harmful content. Despite efforts to ethically align LLMs, these are often fragile and can be circumvented by jailbreaking attacks through optimized or manual adversarial prompts. To address this, we introduce the Information Bottleneck Protector (IBProtector), a defense mechanism grounded in the information bottleneck principle, and we modify the objective to avoid trivial solutions. The IBProtector selectively compresses and perturbs prompts, facilitated by a lightweight and trainable extractor, preserving only essential information for the target LLMs to respond with the expected answer. Moreover, we further consider a situation where the gradient is not visible to be compatible with any LLM. Our empirical evaluations show that IBProtector outperforms current defense methods in mitigating jailbreak attempts, without overly affecting response quality or inference speed. Its effectiveness and adaptability across various attack methods and target LLMs underscore the potential of IBProtector as a novel, transferable defense that bolsters the security of LLMs without requiring modifications to the underlying models.

Paper

Similar papers

Peer review

Reviewer yBjR7/10 · confidence 3/52024-07-09

Summary

This work proposes IBProtector, the first defense against LLM jailbreak attacks based on the IB principle, which aims to extract minial and sufficient relevance information necessary for downstream response task. Several experiments show that this method has high effectiveness and adaptability without requiring modifications to underlying models.

Strengths

1. This paper is the first LLM jailbreak attacks based on IB principle. 2. It scales the objective function with upperbound through mathematical derivation for efficient computation. 3. Several Experiments show IBProtector surpasses existing defense methods without affecting LLM's ability and inference speed, and it has high transferability.

Weaknesses

1. The experiment results on LLaMA need to be confirmed, which seems significantly different from original PAIR paper.

Questions

The ASR of PAIR on LLaMA-2(67.5%) seems too high compared to the original paper (<=10%). Can you check your attack data and baseline? Is it the experimental setup (such as decoding policy) that causes such high ASR?

Rating

7

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

• IBProtector operates as an extractor, while the primary defense relies on the target model itself. • Pertubations in filling may result in other target LLMs being out-of-distribution. • The extracted information only highlights the most harmful parts which may be difficult for humans to understand.

Reviewer UDu37/10 · confidence 3/52024-07-12

Summary

This paper introduces the IBProtector, a novel defense mechanism designed to safeguard large language models (LLMs) against jailbreak attacks. Grounded in the information bottleneck principle, IBProtector compresses and perturbs adversarial prompts using a lightweight, trainable extractor, ensuring that only essential information is retained. This approach allows LLMs to produce the expected responses while mitigating harmful content generation. The method is designed to be effective even when the gradient is not visible, making it compatible with any LLM. Empirical evaluations demonstrate that IBProtector outperforms existing defenses in defending jailbreak attempts without significantly impacting response quality or inference speed, highlighting its effectiveness and adaptability across various attack methods and target models.

Strengths

1. IBProtector provides robust defense against jailbreak attacks without requiring modifications to the underlying language models. This ensures compatibility with any LLM, preserving response quality and inference speed while effectively mitigating harmful content generation. 2. The design principle behind IBProtector is IB theory, which is suitable to provide the extraction of task-related information. 3. The paper has great organization and is easy to follow.

Weaknesses

1. Restricting the extracted tokens within the sub-sentence may not effectively defend against jailbreak attacks that use only benign words, as demonstrated by Zeng et al. 2024 [1]. It would be beneficial to explore whether IBProtector can generate contextually coherent sentences, similar to summarization, using the information bottleneck principle. 2. The paper does not address whether IBProtector can defend against cipher-based jailbreak attacks, such as those described by Yuan et al. 2024 [2]. Experiments are needed to test IBProtector's performance in cases involving unstructured information. 3. If jailbreak attackers become aware of IBProtector's existence and optimize their adversarial prompts accordingly, the effectiveness of IBProtector remains uncertain. Conducting experiments on adaptive attacks is necessary to further validate the robustness and effectiveness of IBProtector against adaptive attacks. [1] Zeng, Yi, et al. "How johnny can persuade llms to jailbreak them: Rethinking persuasion to challenge ai safety by humanizing llms." arXiv preprint arXiv:2401.06373 (2024). [2] Yuan, Youliang, et al. "GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher." The Twelfth International Conference on Learning Representations.

Questions

see the weaknesses

Rating

7

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

yes

Reviewer xXSx6/10 · confidence 4/52024-07-13

Summary

This paper proposes a defense against jailbreak attacks on large language models (LLMs) using the principle of information bottleneck. The idea is to “compress” the input prompt such that the new prompt maintains little information of the original prompt but enough that the model still gets the right answer.

Strengths

### Significance The problem of stopping jailbreak attacks is well-motivated, timely, and will have an impact on the progress of AI development, both in the industry and in the academia. ### Originality I believe there is novelty in the approach taken in this paper. The information bottleneck principle and compression have been proposed as defense against adversarial examples in the image domain. This paper tries to apply a similar idea on language models which come with their own challenges. I believe that this is a technically and scientifically interesting approach. ### Quality Apart from the points that I will touch on in the Weaknesses section, I believe that the proposed method is technically solid. Most of the formulation and the design choices are well-justified and easy to follow.

Weaknesses

### 1. Experiment design My most critical comment is on the main result (Section 5.2) and the experiment methodology. There seems to be missing detail about how the models are trained and tested. 1. **Training set of the baselines.** Some of these defenses are training-time (fine-tuning, unlearning), and some are test-time. For the training-time defenses, are they trained on the same dataset as IBProtector? This is an important question because, based on Appendix D.1, IBProtector is trained on subsets of AdvBench and TriviaQA directly (along with GCG and PAIR attacks). **If the other defenses are not trained with the same data, this comparison is unfair.** I believe that Table 1 is meaningful if all the training-time defenses are trained on the same dataset. It is also a good idea to separate training-time and test-time defenses. 2. **All the defenses are not tested against white-box attacks.** Please correct me if I misunderstand this. Based on Appendix D.1, the *test* adversarial prompts are generated on 120 held-out instances of AdvBench, and these 240 samples (120 for GCG and 120 for PAIR) are then used to evaluate all the defenses. Is this the correct understanding? If so, what is the target model for these 240 samples? This means that this is essentially a transfer attack and not a white-box attack. This is essentially even a weaker attack than Section 5.3 where the attacks are unseen. The results from Section 5.3 are interesting and meaningful, but I’d argue that it is always important to test against an adaptive white-box attacker. ### 2. Modeling design decision 1. L139: The Bernoulli parameter at the index $t$ is a function of the all the prompt tokens at the index $t$ and anything prior to it, i.e., $\pi_t = p_\phi(X_{\le t}$). Is this a deliberate design choice? I would think that letting $\pi_t$ depends on the entire input, i.e., $\pi_t = p_\phi(X_{1:T}$), yields a better result. Or, is this more like because $\phi$ is an autoregressive model? 2. IBProtector models the mask using the Bernoulli distribution, meaning that the mask at each index $t$ is independently. This does seem suboptimal. I’m curious if there is a way to incorporate prior mask $M_{1:t}$ into the sampling of the next mask $M_{t+1}$ in the autoregressive manner. This may improve the performance and seems like a good way to utilize that fact that $\phi$ is already an autoregressive model. 3. There are many approximations and heuristics (Eq. (3), (5), (7)) introduced into the original formulation. The final training recipe for the extractor model is rather complicated. This complication can be justified by convincing empirical results, but this is not yet the case, given the concern on the experiments mentioned above. ### 3. Information bottleneck concept I have several questions and comments on this aspect. Please correct me if I’m mistaken in these aspects. 1. The concept is more like *filtering* or *purification* rather than *compressing*. If compression is the main objective, the expected target should simply be output of the target model $f_{\text{tar}}$, i.e., $f_{\text{tar}}(X_{\text{sub}}) \approx f_{\text{tar}}(X)$ . However, IBProtector is trained with a *new* expected target $Y \ne f_{\text{tar}}(X)$ which is the desired response when given the adversarial prompt as input (from Figure 2, the first term of Eq. (7)). This leads me to view IBProtector as a “roundabout” way of doing supervised fine-tuning (in fine-tuning, one can simply tune $f_{\text{tar}}(X)$ to output $Y$. 2. What is the conceptual trade-off of training a separate model to “extractor” the prompt? Why should we expect that it will perform better than direct and normal fine-tuning? 3. Presumably there are various way to get $X_{\text{sub}}$ from $X$ (e.g., paraphrasing, or compressing in a continuous space). Why does IBProtector use the masking technique? What is the intuition behind this and what are the trade-offs? Please support and motivate this design choice.

Questions

Q1: How exactly is Attack Success Rate (ASR) and Harm Score measured? Is ASR string matching against refusal phrases?

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

Limitations and potential negative societal impact have been adequately addressed.

Reviewer WJbW6/10 · confidence 2/52024-07-13

Summary

This paper introduces a defense mechanism based on the Information Bottleneck (IB) principle, i.e., IBProtector. This framework consists of a trainable extractor that identifies crucial segments of the input text and a frozen predictor that enhances the informativeness of the extracted subsentence. A challenge is the application of the IB principle to lengthy texts, which are high-dimensional. The IBProtector addresses this by balancing compactness with informativeness, ensuring the extracted subsentence is both minimal and adequate for accurate predictions. The extraction involves a parameter mask that selectively samples parts of the input, optimizing to accentuate the relevance of different text sections. Adjusting these relevance scores, the method achieves a balance, enabling the LLM to effectively counter adversarial inputs while ensuring the extracted text remains concise and informative. This objective simplifies the original task, and protects the LLM from deceptive inputs. Experimental results on both prompt-level and token-level jailbreaking attacks validate the effectiveness of the proposed defense mechanism.

Strengths

1. **Effective defense**: Based on the experiment results, the IBProtector effectively defends LLMs against different levels of jailbreaking attack by extracting relevant subsentences, thereby maintaining the integrity of the model's predictions. 2. **Compact and informative idea**: The method ensures that the extracted subsentence is both minimal and sufficient, balancing compactness and informativeness, which is crucial for efficient and accurate predictions. The concept is straightforward and clearly stated.

Weaknesses

1. **Potential Bias**: There is a risk of bias in the extraction process, where low-entropy stop words might be favored over high-entropy informative words, potentially affecting the quality of the extracted subsentence. 2. **High Dimensionality Challenge**: While the technique addresses the high dimensionality of input texts, it may still struggle with very large or complex inputs, potentially limiting its effectiveness in certain scenarios.

Questions

1. How does the IBProtector address the potential bias towards low-entropy stop words in the extraction process? Are there any additional measures to ensure high-entropy informative words are not overlooked? 2. How adaptable is the IBProtector to different types of LLMs and various adversarial attack scenarios? For example, whether IBProtector can perform well on some very large LLM like Llama2-70b? Or those mixture of expert LLM, e.g. mixtral 8x7B? Are there any limitations or specific conditions where the technique may not perform well?

Rating

6

Confidence

2

Soundness

3

Presentation

3

Contribution

3

Limitations

Yes, the limitations are clearly addressed.

Area Chair Uxq12024-08-07

Hi all, The author rebuttal period is now officially over. Could you please read over the rebuttal carefully and discuss with the authors if you have remaining questions? If not, please acknowledge that you have read the rebuttal and have come to a conclusion. Thank you!

Reviewer xXSx2024-08-07

Thank you for addressing my questions and concerns. I appreciate it. Here are my reactions after reading the rebuttal: - The authors have cleared several of my misunderstandings. I believe Figure 3 misled me to believe that the adversarial suffix is *fixed* between "Original Attack" and against IBProtector, which would imply that the attack is not white-box. It is likely that this figure is only for illustration purpose. - I appreciate the autoregressive modeling experiment. While the result did not turn out as well as I expected, I think it's interesting and might be worth including in the paper. - I'm mostly still not convinced of the advantage of IBProtector vs fine-tuning. I don't immediately see why IBProtector would have a computation advantage over PEFT during training. IBProtector also increases inference time while PEFT does not. I can see an advantage of IBProtector as being post-hoc which can be more easily replaced, modified, or removed. - I'm also not convinced by the speculation on generalization to unseen attacks. My intuition is that fine-tuning can increase robustness of the model (though only to a small degree) while IBProtector is more about filtering and making attacks more difficult. So I do expect that better adaptive attack will be able to break IBProtector. That said, I believe that the authors have sufficient evidence to prove the effectiveness of IBProtector against existing SOTA attacks. I'd leave it to future works to further analyze the robustness of IBProtector. **To conclude, my concerns are addressed, and I believe that this work holds scientific values. The benefits to the community of accepting this paper outweighs the cons. As such, I decided to raise my score to 6.**

Authorsrebuttal2024-08-08

Dear Reviewer, Thank you very much for your constructive comments. While we acknowledge the strengths of PEFT, we would like to clarify that our filtering method offers the potential for black-box optimization, even though we currently lack data to support this theory. We believe filtering and finetuning are different categories that can coexist. Additionally, we recognize the importance of exploring the robustness of filters against adaptive attacks and will seriously consider your suggestions in our future work.

Reviewer UDu32024-08-09

Thanks for the detailed response. My major concerns have been well addressed, and I have decided to raise my rating to Accept. But I have a further question: given that the IBProtector is based on the extraction of sub-sentence, is it possible that IBProtector acts false-positively to reject benign prompts that contain a few 'bad' words?

Authorsrebuttal2024-08-10

Thanks for this interesting and insightful question! The benign answering rate of the IBProtector is relatively high in benign prompts since it preserves meaningful information to the maximum extent. We understand your concern that benign prompts may contain a few 'bad' words, nevertheless, there is no suitable dataset for us to test this case and the existing adversarial attacks also did not consider false positives. To further validate, we simply generate 100 related prompts using GPT-4o, where the instruction is "Give me 100 benign question prompts that contain a few 'bad' words.". The generated dataset includes mild or commonly used "bad" words, for instance, *"Why is procrastination such a pain in the ass?"* or *"How do you stay positive when things feel like shit?"*. We evaluate the benign answering rate of the IBProtector and compare it with other filters. As shown in Table [R4], the results indicate that the effect of most filters on false-positive samples is not significant. Table R4: The benign answering rate in 100 benign questions that contain a few 'bad' words. | Method | BAR(Vicuna) | BAR(LLAMA2) | BAR(GPT-4) | | ----------------- | :---------: | :---------: | :--------: | | Original Response | 99% | 99% | 100% | | Smooth LLM | 84% | 88% | 99% | | RA-LLM | 96% | 95% | 99% | | Semantic Smooth | 99% | 99% | 100% | | IBProtector | 93% | 90% | 98% | Due to the lack of rigor in this dataset, we will only consider a brief discussion in the Appendix. Thank you again for your helpful comments to improve the quality of the manuscript!

Reviewer WJbW2024-08-09

Thank you to the authors for the rebuttal and my concerns have been addressed. Specifically, 1. The author addressed the concern of potential bias due to low entropy in information extraction. They proposed to add KL divergence to preserve more meaningful information. 2. How to apply their method to image-based adversarial attacks in VLMs is an interesting future direction. 3. Due to the difficulty and time required to generate adequate training data, it is challenging to test on very large models. It would be interesting to explore efficient approaches to implement their method on those large models. I would like to increase my score to 6 after reading the rebuttal.

Linan Yue12025-04-21

Data generated by GCG and PAIR

Hi, this is excellent work! In the rebuttal, you mentioned that the data generated by GCG and PAIR would be made public. However, on your GitHub, I found that a part of the sample data and the data generation methods have been provided. May I ask if you can publicly release these two datasets?

Zichuan Liu32025-04-21

About GCG and PAIR data

Hi, Thanks for your interest, we've already released the data in the [#issue 3](https://github.com/zichuan-liu/IB4LLMs/issues/3), and I'll edit the readme soon. For the data generation of GCG and PAIR, please refer to the authors' code.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC