Summary
This paper introduces a new benchmark, FB-Bench, designed to evaluate the responsiveness of LLMs to human feedback in real-world scenarios. The authors argue that existing benchmarks primarily focus on single-turn dialogues or multi-turn dialogues with independent user inputs, which fail to capture the complexity of human feedback in iterative interactions. FB-Bench addresses this gap by offering a more nuanced evaluation framework. FB-Bench is the first systematic benchmark focused on evaluating LLMs' responsiveness to human feedback across diverse real-world tasks. It includes 734 curated samples covering eight task types, five types of response deficiencies, and nine types of user feedback. The authors develop a hierarchical taxonomy that organizes human-LLM interactions into three components: user queries, model responses, and user feedback. This taxonomy supports two main interaction scenarios: error correction (where the model must improve its response based on user feedback) and response maintenance (where the model must maintain a correct response despite potentially misleading feedback). The evaluation is conducted using a weighted checklist specific to each sample, with GPT-4o (August) acting as a judge to score follow-up responses based on human-curated reference responses. The paper evaluates 31 popular LLMs using FB-Bench and reveals significant performance discrepancies between error correction and response maintenance scenarios. Notably, most LLMs perform better in error correction than in maintaining correct responses when faced with misleading or irrelevant feedback.
Strengths
Originality:
While multi-turn evaluation has been attempted before, the authors present a new perspective focusing on error correction and responses maintenance, which seem less explored.
The authors use a checklist when doing LLM judge evaluations for more finegrain judgments.
Clarity:
The paper is well organized and easy to read, with nice structure and clean figures.
Significance:
The authors find an interesting comparison between error correction and responses maintenance (which inherently seem a bit in tension). The authors find most models have superior error correction performance, suggesting a current weakness in LLMs.
The benchmark itself fills an existing hole for evaluating models on multi-turn conversations.
Weaknesses
Out of distribution first turn:
The first turn of the data is collected from some less human preferred response rather than the model tested itself. Naturally, this means the first turn is already out of distribution for the model. It is unclear if this invalidates the real-world applicability of these results. For example, the paper finds Claude-3.5-Sonnet has very low response maintenance (relatively). Would this still happen if Claude-3.5-Sonnet was asked to maintain its own answer? Take another example, Phi-3-small-8k-instruct has very high response maintenance, but low error correction. This would be desired behavior if the model’s own responses were usually correct. However, if the model's response is not fully correct (which is more likely because the model is very small) this response maintenance is actually a negative when deployed in the real world.
Moreover, just with respect to the pure out of distribution nature of the first turn (you are basically putting someone else’s words in the LLM’s mouth), it may be helpful to see an experiment that is single turn. In this case, the model being tested is given the original user query, and the first response in which the model is asked to correct/maintain the response. This way, no out of distribution tokens are inserted into the assistant section of the sequence. If results in this experiment are similar, it would be sufficient to show that the out of distribution first turn is, at least mechanically, not an issue.
LLM Use in Data Curation:
In lines 271-272 the authors state that GPT-4 was used to find the cause of dissatisfaction when the user’s expectations were not met. However, this feels in contradiction to the conclusion of the paper that LLMs are weak for response maintenance. That is, the human’s expectations could have been wrong or unreasonable, in which the GPT-4 annotator may hallucinate some incorrect cause of dissatisfaction via the same sycophant behavior this paper finds almost every LLM has. Annotation process is not explained in enough detail to convincingly rule out this possibility.
Analysis:
While the results for models are interesting, the analysis focuses on superfluous information rather than deeper insights. For example, the authors write on line 379 that “This disparity [between error correction and response maintenance] is more pronounced among LLMs developed outside of China”. This claim seems a bit out of place, and is not supported with concrete numbers (especially because the benchmark seems to be in Chinese, which will obviously yield better results on models trained for this language, and is a huge confounding variable). Moreover, analysis like this is not particularly tied to the core exploration of error correction vs response maintenance. As a reader, error correction vs maintenance feels more analogous to precision vs recall. Poor performance overall (like Claude-3.5-Sonnet) seems like a calibration issue, not an inherent flaw of the model. For example, any model could achieve 100% success on response maintenance with a system prompt that tells it NEVER to change its initial response. It seems error correction and response maintenance are something that could be calibrated. Looking at Figure 4, it seems “evenly calibrated” models will lie approximately on y=x. However, this assumes that error correction and response maintenance occur at the same rate in the real world, which may not be true. For example, the “smarter” a model gets, the more it should stick to response maintenance rather than error correction, since it can assume it is more skilled with respect to the human user giving feedback. This point connects back to my original critique with out of distribution first turns covered above— we can never know the real aggregate performance of a model without first understanding how often the model will get the first turn incorrect or unsatisfactory. For real-world applicability, the authors should at least consider the tension between error correction and response maintenance in their core analysis. For example, what happens when we use a system prompt that always forces error correction? What about the opposite? Will these get top accuracy in each category if used in conjunction with a suitably strong model?
LLM Judge:
While the usage of the checklist judge is interesting, the authors do not verify the judge's accuracy. This on top of the fact, the checklist itself is GPT-4 generated and seemingly unverified. For example, does the ranking change if we simply change the judge to a different strong model? Why was GPT-4o-08-06 selected as a judge? The authors should have some level of verification for judge quality and simultaneously use this to justify their judge choice.
Questions
Most big questions ended up in the weakness section, however some desired clarifications are shown below:
On lines 270-275, the authors state that GPT-4 was used for annotation. Which GPT-4 is this (0314, 0613, or some turbo version)? Why was GPT-4 used instead of new models which seem to perform better?
On line 473, the authors criticize that data in previous multi-turn benchmarks are synthesized by LLMs. However, it appears that LLMs are used to synthesize a significant portion of FB-Bench’s data (lines 272-275). Why does this same criticism not apply to FB-Bench?
Figure 6 is interesting, but the very light colors on the white background are a bit hard to see. Perhaps slightly darker colors, less line opacity, and points on the axis would make it slightly more readable.
Multiple times in the paper the authors reference “advanced” models. It is a bit ambiguous what “advanced” encompasses. A more explicit definition would be helpful.
In section 2.4, the authors explain their data is from “an online chat service and human preference data”. Could the author please give more detailed information on what these exactly are, why they are reputable, and give examples from the data sources?
In line 261, the authors say they use a heuristic to identify target data, but never specify in any detail what this heuristic is, or why it works.
In line 273, the authors say GPT-4 is used to generate the judgment checklist. How are the weights for the checklist decided? How is the checklist verified? Does removing the checklist significantly change final ranking scores?