Summary
The issue of robustness in neural networks has gained paramount importance over recent years. With the surge in performance of Large Language Models (LLMs) across various domains, the question of their robustness becomes increasingly pertinent. In this work, existing datasets for robustness evaluation are classified into two categories: closed-question tasks such as GLUE and open-ended tasks like TriviaQA. However, with the evolving generative capabilities of LLMs, there is a growing concern that these tasks may no longer serve as appropriate benchmarks. To address this gap, this work introduces a GPT-based evaluation framework, named the "Reward Model for Reasonable Robustness Evaluation" or TREvaL. This framework is designed to assess the robustness of LLMs in the context of open questions. Instead of focusing purely on accuracy or similarity, TREvaL emphasizes the correlation between the content generated by the model and the selected open prompts. This is achieved by leveraging trained reward models to serve as evaluative judges. The experimental setup encompasses two foundational LLM families, Beaver and Llama2, spanning various model sizes. The primary adversarial attack methods utilized are word-level perturbations, encompassing strategies like word swapping, synonym substitution, and common misspellings. The findings underscore that the LLMs examined in this study exhibit a discernible gap in robustness. Also, the impact of parameters on harmlessness robustness seems to be slighter. Notably, the fine-tuning process appears to compromise the model's robustness. To further substantiate this observation, the paper presents loss landscape visualizations for different model stages, reinforcing the central findings.
Strengths
(1) The paper is well-motivated and aims to rectify prevailing limitations in current benchmarks and evaluation protocols for LLM robustness. Considering the recent advancements in LLMs, research focusing on robustness evaluation stands to benefit both the academic community and practical applications greatly.
(2) The paper introduces a new robustness evaluation framework tailored specifically for LLMs. By emphasizing open questions and leveraging trained reward models, it intends to address and potentially overcome the limitations inherent in existing benchmarks.
(3) The experiments encompass three types of word-level perturbations and evaluate two series of open-source LLMs. By assessing robustness across different stages and model sizes and providing a visual representation through the loss landscape, the authors offer a comprehensive and holistic evaluation.
(4) The provided figures, tables, and mode details in the appendix effectively aid in clarifying the main ideas of this work, making them easily understandable.
Weaknesses
(1) There seems to be a notable absence of a side-by-side comparison between TREvaL and existing benchmarks. Although some limitations in the existing benchmarks are mentioned briefly, it is not clear without such a comparison whether the proposed framework can address these limitations. Establishing a direct contrast would solidify the claims regarding the superiority or distinctiveness of TREvaL in the domain of LLM robustness evaluation.
(2) Following this point, the paper could benefit from a more comprehensive literature review, especially considering the plethora of recent studies centered on the robustness of LLMs [1]. The connections and distinctions between this work and prior research in the field are not thoroughly explored, which might leave readers without a complete contextual understanding of the contributions in the broader academic landscape.
(3) TREvaL incorporates trained reward models in the evaluation process, yet there is no explicit motivation or justification for this choice. A more detailed exposition of their role, significance, and advantages over other potential evaluative metrics would enhance the clarity and persuasiveness of TREvaL.
(4) The technical innovations appear somewhat incremental. For example, the reliance on just three basic types of word-level perturbations feels somewhat rudimentary, given the wide spectrum of possible textual adversarial attacks. While the focus on evaluating robustness in the context of open questions and across different fine-tuning stages is commendable, other aspects of the paper do not seem to introduce significantly new perspectives or methods to the field.
[1] Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluations
Questions
(1) Could you provide more insights comparing TREvaL directly with existing benchmarks? Otherwise, how can we confirm that the proposed framework addresses the limitations of current benchmarks more effectively?
(2) What motivated the choice to integrate a reward model within the evaluation framework? An immediate concern is whether the reward model itself is susceptible to the same perturbations, potentially compromising the integrity of the evaluation.
(3) Several definitions and statements appear to stand without clear foundational support. For instance, is robustness specifically defined as "the rate of performance decline under potential perturbations"? Is it accurate to categorize benchmarks into three distinct classes, as suggested? Furthermore, the definitions of terms like "helpfulness" and "harmlessness" robustness remain unclear. Also, how widespread is the acceptance of using a loss landscape as a tool for characterizing robustness?
Rating
3: reject, not good enough
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.