Understanding and Enhancing the Transferability of Jailbreaking Attacks

Summary

This paper focuses on the transferability of jailbreaking attacks. By quantitatively studying the model's intent perception based on Llama-2-7B/13B, this paper concludes that the inconsistent effectiveness of the adversarial prompts across source and target LLMs is due to the inherent distributional dependency. This paper also proposes a novel attack method named PiF.

Strengths

1. This paper provides a plausible explanation for the transferability of jailbreaking attacks, backed up with a demonstrative example shown in Figure 2. I believe this is the right direction for transferable jailbreaking attack research. To better support the claims in Section 3, I suggest providing more quantitative results in future revisions. It is also notable that the presentation of this paper is very good. The color scheme is carefully chosen. The tables and the LaTeX boxes are well-made. It is a pleasure to read this paper.

Weaknesses

1. Some results in this paper are merely demonstrative, e.g., those shown in Figures 1 and 2. As its title suggests, this paper works on both understanding and enhancing the transferability of jailbreaking attacks. The effort put into "understanding" is much less than those on "enhancing". More quantitative results would greatly help support the claims in Section 3. 2. Perspective III, i.e., the distribution dependency, is trivial. It is not surprising that the high-importance regions are dependent on the model's parameters. As the term distribution dependency is emphasized many times (even in the abstract!), the readers would expect a more significant result than those in Section 3.3. 3. Evaluation. + In Table 2, only GCG and PAIR are compared. I suggest comparing some other popular jailbreaking attacks, e.g., AutoDAN [1] and ReNeLLM [2]. + In Line 395, it is claimed that only the outputs are accessible. However, GCG is a white-box attack method that requires the access to model's parameters, leading to a contradiction. It is also very confusing how it is possible to attack GPT-4 using GCG. I suppose GPT-4 is a close-sourced model which does not meet the requirement of GCG. + I cannot find the results about "enhancing" the transferability of the jailbreaking attacks. There are also some minor issues. 1. It seems that Figure 1 does not illustrate the main idea of this paper, which is slightly misleading. 2. In Table 1, the existing jailbreaking attacks are categorized into model/token/prompt-level and hand-crafted. It seems that the "model-level attacks" refers to the white-box attacks in the literature. Could the authors please provide some references for this kind of taxonomy? **Reference** [1] AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models. Xiaogeng Liu, Nan Xu, Muhao Chen, Chaowei Xiao. [2] A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily, Peng Ding, Jun Kuang, Dan Ma, Xuezhi Cao, Yunsen Xian, Jiajun Chen, Shujian Huang.

Questions

See the weakness part, especially the evaluation.

Rating

6

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Ethics concerns

No ethics review is needed.

Summary

This paper proposes a simple but effective Perceived-importance Flatten (PiF) attack against LLMs by dispersing the model's focus across neutral-intent tokens, thereby reducing its attention on malicious-intent tokens. Experiments from various perspectives illustrate that, compared to existing baselines like GCG and PAIR, PiF shows high transferability, efficiency, and attack success rate (ASR).

Strengths

1. The paper introduces a novel and efficient method to reduce LLMs' focus on malicious-intent tokens, thus bypassing the safety alignment. 2. The PiF method demonstrates high transferability across various target LLMs while maintaining a consistently high ASR. 3. Compared to existing baselines, PiF uses fewer tokens in the adversarial prompt and requires less optimization time.

Weaknesses

1. The paper lacks comparisons with more recent baselines [1, 2, 3], which have been shown to be more effective and efficient than GCG and PAIR. 2. The authors should clarify the detailed implementation of the perplexity and instruction filters used in the paper, either in Section 5.2 or the appendix. For example, in Table 3, while PiF's perplexity is lower than GCG's, it is 9x higher than PAIR's, suggesting that a suitable perplexity threshold could filter out PiF's prompts. Listing the performance of normal text under these filters would help to justify PiF's stealthiness. 3. The hyper-parameter settings for GCG and PAIR should be listed in Section 5 or the appendix so that reviewers can assess the fairness of the comparisons. 4. In addition to GPT-4, the authors should include tests with more recent models such as GPT-4o, Claude-3, and the Gemini-1.5 series. [1] Zeng, Yi, et al. "How johnny can persuade llms to jailbreak them: Rethinking persuasion to challenge ai safety by humanizing llms." ACL. 2024. [2] Liao, Zeyi, and Huan Sun. "Amplegcg: Learning a universal and transferable generative model of adversarial suffixes for jailbreaking both open and closed llms." COLM. 2024. [3] Shen, Guangyu, et al. "Rapid optimization for jailbreaking llms via subconscious exploitation and echopraxia." arXiv preprint arXiv:2402.05467 (2024).

Questions

1. In the performance evaluation, did the authors generate only one adversarial prompt for each malicious request, or multiple adversarial prompts are generated for a single malicious request? 2. In real-world applications, user prompts may be paraphrased by software for better inference. Is the PiF attack robust to such paraphrasing?

Rating

6

Confidence

4

Soundness

3

Presentation

2

Contribution

2

Thanks for The Response

Dear Authors, Thank you for your prompt response. However, based on your additional results, I have the following concerns: 1. Considering the low AHS against Claude and Gemini models, I question whether the outputs from these models are still harmful. 2. Gemini models are reported to have output filters that prevent potentially harmful responses. Why does PiF achieve such a high ASR against the Gemini-1.5-flash, even surpassing the AHS observed against the Claude model? 3. Regarding the paraphrasing experiments, could you further show how the rewrite prompts alter the malicious prompts generated by PiF and other baselines? This would provide additional justification for the stealthiness of PiF. Additionally, here are several questions related to PiF's design: 1. When calculating perceived importance, how is the change in logits at the [MASK] position measured? Do you only consider changes to logits for specific tokens (e.g., "legal" and "illegal"), or do you consider the entire logits distribution at the [MASK] position? 2. How do you generate the top-M candidates for replacement in Stage II? Are these M candidates sampled by a model, and if so, how is the sampling performed? 3. How is sentence similarity calculated in Stage III? Is it based on another small model, or do you use specific metrics (e.g., BLEU score)? 4. If a replacement does not meet the sentence similarity threshold, does PiF return to Stage I to select a new token to replace, or does it return to Stage II? If PiF cannot find a suitable replacement that satisfies the threshold, will the process terminate, or are there alternative procedures to solve this case? Best Regards, Reviewer RdHb