On the Worst Prompt Performance of Large Language Models

Summary

The authors propose a new benchmark to study the robustness of LLMs to prompt variations. Different from previous work, this paper mainly focuses on semantically equivalent prompts rather than taks-level instructions. The experiments demonstrate that many popular LLMs are sentitive to the form of prompts. Moreover, it is difficult to detect the worst prompt and improve the worst prompt performance.

Strengths

1. Research Value: The paper tackles a highly relevant problem that has substantial implications for the practical deployment of LLMs. The issue of prompt sensitivity is critical and the paper provides valuable insights into this domain. 2. Clarity of Motivation: The authors have effectively articulated the motivation behind their work. The introduction and the problem statement are clear, setting the stage for the reader to understand the importance of the study.

Weaknesses

1. Lack of Solution: While the paper adeptly identifies the problem of prompt variability and its impact on LLM performance, it falls short of proposing an effective solution to mitigate this issue. The research would benefit from the inclusion of potential remedies or strategies to enhance prompt robustness. 2. Inconsistency in Description: There appears to be a discrepancy between the textual description and the visual representation (Figure 1) regarding the focus on task-level instructions versus case-level inputs. Clarification on this point is needed to avoid any confusion for the readers. 3. Limited Scope of Experimentation: The experiments, while extensive, do not include testing on more advanced models such as GPT4. The performance of the proposed benchmark on state-of-the-art models would provide a more comprehensive understanding of the robustness of the latest LLMs. 4. Language Specificity: The study seems to be limited to English language prompts. Given the global application of LLMs, it would be beneficial to extend the experiments to include other languages to assess the generalizability of the findings. 5. Prompt Diversity Validation: The paper does not explore whether incorporating a diverse set of prompts in the SFT stage could potentially improve model's worst prompt performance. An analysis of this nature would add depth to the paper's contribution.

Questions

See Weaknesses.

Rating

6

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors have adequately addressed the limitations.

Summary

This paper introduces a new benchmark, ROBUSTALPACAEVAL, that contains semantically equivalent queries of diverse real-world tasks, and uses it to conduct extensive experiments on ChatGPT and six open-source LLMs. It highlights variability in model performance and difficulties in predicting the worst prompt.

Strengths

- This work introduces a new benchmark focused on diverse real-world user queries rather than task-level instructions, which more closely mirrors real-world scenarios. - It conducts comprehensive evaluations across several LLMs and showcases variability in model performance across different prompts. - They tried several methods to improve the worst prompt and show the limitations of current method.

Weaknesses

- The methodology utilizes gpt4_turbo as the evaluator and the reference model for outputs, which might introduce biases. A discussion of this potential bias should be included. - The ROBUSTALPACAEVAL benchmark contains 10 paraphrases per given query. It’s unclear how performance disparities between the worst and best prompts would change with an increased number of paraphrases. This might also affect the robustness of cases with significant performance gaps. - While the study explores several LLM families including Llama, Gemma, and Mistral, it includes only one version of ChatGPT and does not specify which version. Whether the prompts are uniform across different ChatGPT versions is unknown.

Questions

- How is prompt perplexity calculated? - Are there any common features among the worst-performing prompts within the same model family? The paper points out the difficulty in predicting the worst prompts, both model-independently and with model access. Given that the overlap rate of model-agnostic worst-4 prompts reaches about 40% in the Gemma family, an analysis comparing the features of the worst and best prompts could be insightful.

Rating

6

Confidence

3

Soundness

3

Presentation

3

Contribution

2

Limitations

The paper includes a limited selection of LLMs, and the challenge of effectively improving the worst-performing prompts remains unresolved.

Summary

In this paper, the author(s) propose a benchmark for prompt performance of large language models. In particular, the author(s) leverage GPT4 to generate variants of prompts and hence constitute a dataset. Afterwards, the author(s) evaluate the performance of these generated prompts, and explore the identification and improvement of worst cases.

Strengths

- This study acknowledges the variation and diversity of prompts in real-world, and adds value to the prompt engineering for LLMs. - The proposed benchmark is applied to multiple LLMs. Thorough comparison analysis is conducted. - The paper is well-organized, and the language is technical yet understandable for readers with domain knowledge.

Weaknesses

- The methodology and result analysis of this study can be elaborated. - Result visualization can be improved for better readability.

Questions

- Section 2: A related work can be added and analysed. Schulhoff, S., Ilie, M., Balepur, N., Kahadze, K., Liu, A., Si, C., ... & Resnik, P. (2024). The Prompt Report: A Systematic Survey of Prompting Techniques. arXiv preprint arXiv:2406.06608. - Please elaborate the motivation and result analysis for exploring the hidden states in Section 4.2. - I wonder if the author(s) consider sharing the dataset used in this project? - The author(s) can consider specifying the selection criteria of LLMs used in experiments. - Please consider using bold in tables to highlight the noteworthy data. - It will be interesting to summarize the features/patterns of worst prompts for different LLMs. - Please adjust the place of tables and figures to improve readability. For instance, Figure 3 is on Page 6 while it is described on Page 7.

Rating

6

Confidence

3

Soundness

3

Presentation

4

Contribution

3

Limitations

- The author(s) note that the research can be extensive by including more LLMs, and applying alternative approaches to identify worst prompts.

Summary

This paper proposes a new benchmark, RobustAlpacaEval, a benchmark with semantically equivalent queries. The authors evaluate performance as the worst performance across all the prompts and show that many prompt consistency methods have a limited improvement on this benchmark. The paper also shows that it is difficult to predict which prompts will have a poor performance.

Strengths

- The paper has a variety of key findings (Section 3.2). First, the authors find that there is a large gap between worst and best performance. Second, the authors find that scaling does not help robustness. Third, the authors find that there is little agreement between various models on which prompts are harder and/or easier. - The experiments are thorough and well conducted. - The findings open up doors for future research and efforts to make models more robust.

Weaknesses

- The variance of a model to different prompts has been studied several times in the literature (for example, see [1]), and it is unclear to me how a benchmark measuring worst-case performance would be used. I would appreciate if the authors provided more clarity on how they envision that this work could shed insight and facilitate future efforts. - In Section 5, only Llama models are considered, which makes it unclear whether the insights discovered also apply to other models or are unique to Llama. - The paper would be much stronger if the authors proposed a method to mitigate the phenomenon demonstrated in the paper, or if the authors gave a detailed explanation of why the identified phenomenon occurs. [1] https://arxiv.org/abs/2401.03729

Questions

- Can the authors suggest some potential directions for resolving the issue raised in the paper? - When creating a benchmark like this, the number of prompts can be increased arbitrarily, and the accuracy could be extremely low if the right prompts are chosen. Similarly, one could create an adversarial setting to test the robustness of each model. Why did you design the benchmark in this way? - How do the authors envision that this benchmark be used? For example, should all future models be forced to evaluate on RobustAlpacaEval? Because there are many more prompts for each sample, this would lead to a multiplicative factor on the cost for inference.

Rating

5

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

Yes

Summary

This paper studies the worst performance an LLM can have on input queries by testing on paraphrases of each query and report the original, worst, best, and average performance. It has been found that there is a large gap between the best and worst performance. It is then found that there is no particular "worst prompt" among all models or even models from the same family. Furthermore, model-dependant features like are also not useful in identifying the worst prompt. Finally, authors explored a few approaches to try to improve the "worst performance". In particular, voting seems effective in improving the lower bound at the cost of harming the upper bound.

Strengths

- This paper discusses an important question to study, that is the lower bound performance of LLMs on different prompts. It is interesting to know that identifying the prompt that leads to bad performance is difficulty by looking at the prompts alone or using an LLM itself.

Weaknesses

- I think the definition of the "worst" or "best" performance is not clearly stated (line 121). If I understood correctly, the worst performance will monotonically drop if we have more paraphrases of each query. This is linked to the question below and I hope that the authors can clarify on this. - Using paraphrases to study prompt variation could introduce errors and paraphraser/LLM preference mismatch. It might be questioned that Table 1 and Table 4's average performance is almost always lower than the original performance. Does this mean that the paraphrasing process has introduced errors, domain shifts, etc?

Questions

- I wonder if you could provide a clear and reproducible definition of the worst and best performance in mentioned line 121. From the writing, I think regarding whether a query is correctly responded by the model: worst=all(prompt1, prompt2, ...) and best=any(prompt1, prompt2, ...). Is this correct? - If this is the case, I think the more paraphrases we have, the higher we get for the best performance and vice versa for the worst performance. This might make the number of paraphrases a bit arbitrary. The worst-best gap might also be inflated. - Throughout the paper, starting with line 15, I think the phrase "worst prompt" is not precise---we cannot call them good or bad per se unless proven. I think that you mean the prompt paraphrase that leads to the worst performance.

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

The paper touches on limitations like not experimenting enough models and enough techniques in finding the "worst prompt".