Summary
**Advantages**
The paper introduces a novel multi-turn jailbreak strategy termed Jigsaw Puzzles (JSP), designed to test and expose the vulnerabilities of Large Language Models (LLMs) when confronted with fractionated queries. JSP bypasses the inner guardrails of LLMs by segmenting harmful questions into harmless fragments, which are then concatenated together to induce the models to generate harmful content.
JSP demonstrates a high success rate of 93.76% across 189 harmful queries on five advanced LLMs, including Gemini-1.5-Pro, Llama-3.1-70B, GPT-4, GPT-4o, and GPT4o-mini, highlighting the safety risks inherent in multi-turn interactions with LLMs.
****
**Disadvantages**
The method is simple yet effective and is quite interpretable. However, In the experimental design, there are several shortcomings:
* I'm confused regarding the implementation of the "Standard Prompting" baseline in Table 1. It is ambiguous whether the baseline merely employs stage 1 for query rewriting or directly inputs the queries to the model without rewriting.
* The authors have only compared JSP with some single-turn attack methods (PAP and PAIR). Since JSP is inherently a multi-turn attack method, and multi-turn attacks are generally more effective than single-turn attacks, a fair comparison should be made with multi-turn attack baselines, such as those referenced in [1,2], to demonstrate the effectiveness of JSP. The authors are encouraged to include these comparisons to provide a more comprehensive assessment of JSP's capabilities.
* The robustness of JSP against existing jailbreak defense methods has not been adequately tested. Although the authors have tested JSP against two system-prompt-based defense approaches, these tests do not constitute universally recognized and effective defense baselines. Based on my understanding of JSP, I suggest the authors test JSP against the following three defenses:
* Gradient Cuff [3]: The results show that ASR(a) is significantly lower than ASR(q), indicating that the model's responses to JSP queries still have high uncertainty (meaning sometimes it refuses and sometimes it does not). Thus, the authors need to compare the effectiveness of JSP in breaking through Gradient Cuff defenses (which defend against jailbreak attacks based on response uncertainty).
* Self-Reminder [4]: While JSP can evade the built-in safety checks of LLMs, the Self-Reminder method focuses on restricting LLM outputs no matter the harmfulness of user inputs. Therefore, the authors need to test the effectiveness of JSP against defenses employing the Self-Reminder system prompt.
* Perplexity (PPL [5] ): JSP splits sentences and words, making user input less coherent. It would be interesting to see whether JSP can bypass defenses based on text perplexity detection.
* Safe-Decoding [6]: Safe decoding utilizes a safety-aware expert LLM to assist in decoding; therefore, the authors should explore whether JSP methods can still induce LLMs to output harmful content when an external expert LLM controls the decoding process.
[3,5,6] required the LLMs to be open-source, so the authors only need to conduct tests on Llama-3.1-70B and report the results. [4] are expected to test on all LLMs.
****
**References**
[1] Multi-Turn Context Jailbreak Attack on Large Language Models From First Principles. Xiongtao Sun, Deyue Zhang, Dongdong Yang, Quanchen Zou, Hui Li.
[2] Many-shot Jailbreaking. Anthropic.
[3] Gradient Cuff: Detecting Jailbreak Attacks on Large Language Models by Exploring Refusal Loss Landscapes. Xiaomeng Hu, Pin-Yu Chen, Tsung-Yi Ho
[4] Defending ChatGPT against jailbreak attack via self-reminders. Yueqi Xie, Jingwei Yi, Jiawei Shao, Justin Curl, Lingjuan Lyu, Qifeng Chen, Xing Xie Fangzhao Wu.
[5] Baseline Defenses for Adversarial Attacks Against Aligned Language Models. Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping-yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, Tom Goldstein.
[6] SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding. Zhangchen Xu, Fengqing Jiang, Luyao Niu, Jinyuan Jia, Bill Yuchen Lin, Radha Poovendran.
Strengths
**Strengths 1** The paper introduces a novel multi-turn jailbreak strategy termed Jigsaw Puzzles (JSP), designed to test and expose the vulnerabilities of Large Language Models (LLMs) when confronted with fractionated queries. JSP bypasses the inner guardrails of LLMs by segmenting harmful questions into harmless fragments, which are then concatenated together to induce the models to generate harmful content.
**Strengths 2** JSP demonstrates a high success rate of 93.76% across 189 harmful queries on five advanced LLMs, including Gemini-1.5-Pro, Llama-3.1-70B, GPT-4, GPT-4o, and GPT4o-mini, highlighting the safety risks inherent in multi-turn interactions with LLMs.
**Strengths 3** The method is simple yet effective and is quite interpretable.
Weaknesses
**Weaknesses 1** The authors have only compared JSP with some single-turn attack methods (PAP and PAIR). Since JSP is inherently a multi-turn attack method, and multi-turn attacks are generally more effective than single-turn attacks, a fair comparison should be made with multi-turn attack baselines.
**Weaknesses 2** The robustness of JSP against existing jailbreak defense methods has not been adequately tested. Although the authors have tested JSP against two system-prompt-based defense approaches, these tests do not constitute universally recognized and effective defense baselines.
Questions
**Question 1**: I'm confused regarding the implementation of the "Standard Prompting" baseline in Table 1. It is ambiguous whether the baseline merely employs stage 1 for query rewriting or directly inputs the queries to the model without rewriting.
**Question 2**: Could the author compare JSP with some multi-turn jailbreak baselines? Such as [1] and [2].
**Question 3**: Is JSP robust against universally recognized and effective defense baselines? The author should evaluate JSP's effectiveness against the [3,4,5,6].