Accelerating Greedy Coordinate Gradient and General Prompt Optimization via Probe Sampling

Summary

The paper proposes a method to accelerate discrete prompt optimization algorithms. In addition to the target model, a smaller draft model is used to reduce the number of candidates to be evaluated by the target model based on an agreement score between the two models. When applied to Greedy Coordinate Gradient and trained with the open-source LLMs Llama2-chat-7b and Vicuna-7b, the approach achieves a better runtime while maintaining the attack success rate (ASR) on AdvBench. The method is also applied to other prompt optimization algorithms such as AutoPrompt, ADE and AutoDAN.

Strengths

- The problem studied in the paper is important. The method makes finding prompts for very large models more efficient. - The method is simple and general enough that it can be applied to any discrete prompt optimization algorithms that rely on LLM candidate evaluation. - Except for some minor typos, the paper is well-presented and is easy to read.

Weaknesses

- Transferability is a very important aspect of adversarial attacks but it is never mentioned in the paper. - The related work section lacks a proper presentation of discrete prompt optimization algorithms. Also, the authors did not mention other prior work that also use a pair of models for discrete prompt optimization. Minor issues: The use of the term "Probe sampling" is inconsistent across the paper. Sometimes it is not emphasized and in lower case. line 310: "In this paper, we propose an algorithm called Probe sampling that can ..."

Questions

- Does this method affect the transferability of adversarial attacks to other LMs (for example from Llama2-chat-7b to Vicuna-7b) compared to plain GCG? - What is the effect of the filtered set size and the draft model (size, similarity to target model) on transferability? - GCG operates at the token level and the draft model has a different tokenizer. What is the effect of the draft model tokenizer on the resulting suffix? also on transferability? (are tokens common to both tokenizers chosen or is the method completely oblivious to tokenization differences)

Rating

8

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

The authors have included a section on limitations and potential negative social impact.

Summary

The authors propose using a significantly smaller draft model compared to the target LLM to filter candidate suffixes, thereby accelerating the training process of GCG-based algorithms. The results demonstrate a faster training speed with enhanced ASR.

Strengths

1. I appreciate the innovative approach of utilizing another LLM to filter suffixes to assist in the attack on the target LLM. 2. The paper is well-organized and written in a formal academic style.

Weaknesses

1. As mentioned on line 82, "one needs to perform B forward passes." In my opinion, these B candidates can be sent to the LLM together within a batch. By fully utilizing the GPU's parallel technique, it actually does not need to take B times the amount of time. Can an LLM with a smaller batch size achieve faster speeds? If not, I think the author should illustrate why the proposed method can speed up. 2. Since the draft model is not optimized, how to guarantee that the draft model and the target model align well, i.e., $\alpha$ is significantly larger than $0$? In my opinion, GCG will generate meaningless words, and the function of the draft model seems to be to check the meaning of these suffixes and discard the meaningless ones. I encourage the authors to explain the the meaning of alignment between the draft model and the target model. 3. While the GCG is a popular method and the $6x$ speed is impressive, there are another methods like advprompter [1] which has faster speed and higher ASR. I encourage authors to compare the proposed method with more efficient methods instead of GCG. [1] Paulus, A., Zharmagambetov, A., Guo, C., Amos, B., & Tian, Y. (2024). Advprompter: Fast adaptive adversarial prompting for llms. arXiv preprint arXiv:2404.16873.

Questions

1. The method Amplegcg [2] achieve better performance than GCG, can the proposed method be applied to speed this traning process? [1] Liao, Z., & Sun, H. (2024). Amplegcg: Learning a universal and transferable generative model of adversarial suffixes for jailbreaking both open and closed llms. arXiv preprint arXiv:2404.07921.

Rating

4

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

none.

Summary

This paper presents a novel algorithm called "Probe sampling" to accelerate the Greedy Coordinate Gradient (GCG) method for optimizing adversarial prompts against large language models (LLMs). The key idea is to use a smaller "draft" model to filter out unpromising candidate prompts, reducing the number of expensive evaluations needed on the full target LLM. The method dynamically determines how many candidates to filter based on measuring agreement between the draft and target models on a small probe set. Experiments show Probe sampling achieves 3.5-6.3x speedups on benchmark datasets while maintaining or improving attack success rates. The technique is also shown to accelerate other prompt optimization and LLM attack methods.

Strengths

- Overall: The paper looks solid and the 5.6x runtime improvement for GCG looks compelling. Also, it’s nice to see speed-ups for AutoDAN and prompt learning methods like AutoPrompt and APE. The idea of reducing computation by resorting to a smaller draft model - specifically for forward passes that consume the most time - makes a lot of sense. - Quality: The empirical evaluation is thorough, testing on multiple datasets and model types. The ablation studies and analysis provide good insight into the algorithm's behavior. The speedups achieved are substantial and practically meaningful. - Clarity: The paper is generally well-written and easy to follow. The algorithm is well explained, and the experimental setup and results are presented in a clear and organized manner.

Weaknesses

- It would be good to comment on the other existing approaches to speed up GCG like Fast Adversarial Attacks on Language Models In One GPU Minute and Making a SOTA Adversarial Attack on LLMs 38x Faster. Since they were out in February/March, probably it’s still fine to treat them as concurrent work, but it would be good to discuss the differences to better contextualize your work.

Questions

No.

Rating

7

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors provide a reasonable discussion of limitations. One additional limitation that would make sense to discuss is the high memory requirement for gradient-based red-teaming methods like GCG, which effectively limits the experiments to smaller models (i.e., only up to 7B parameters).

Follow-up comment

Thanks for the further clarifications. They address my (minor) concerns. I increase my score from 6 to 7.