Evaluating LLMs at Detecting Errors in LLM Responses

With Large Language Models (LLMs) being widely used across various tasks, detecting errors in their responses is increasingly crucial. However, little research has been conducted on error detection of LLM responses. Collecting error annotations on LLM responses is challenging due to the subjective nature of many NLP tasks, and thus previous research focuses on tasks of little practical value (e.g., word sorting) or limited error types (e.g., faithfulness in summarization). This work introduces ReaLMistake, the first error detection benchmark consisting of objective, realistic, and diverse errors made by LLMs. ReaLMistake contains three challenging and meaningful tasks that introduce objectively assessable errors in four categories (reasoning correctness, instruction-following, context-faithfulness, and parameterized knowledge), eliciting naturally observed and diverse errors in responses of GPT-4 and Llama 2 70B annotated by experts. We use ReaLMistake to evaluate error detectors based on 12 LLMs. Our findings show: 1) Top LLMs like GPT-4 and Claude 3 detect errors made by LLMs at very low recall, and all LLM-based error detectors perform much worse than humans. 2) Explanations by LLM-based error detectors lack reliability. 3) LLMs-based error detection is sensitive to small changes in prompts but remains challenging to improve. 4) Popular approaches to improving LLMs, including self-consistency and majority vote, do not improve the error detection performance. Our benchmark and code are provided at https://github.com/psunlpgroup/ReaLMistake.

Paper

References (74)

Scroll for more · 38 remaining

Similar papers

Reviewer BGkA6/10 · confidence 3/52024-05-10

Summary

In their paper, the authors introduce an error detection benchmark called ReaLMistake. This benchmark evaluates four distinct types of errors—reasoning correctness, instruction-following, context-faithfulness, and parameterized knowledge—made by large language models (LLMs) across three selective natural language processing (NLP) tasks. The results reveal several key findings: 1) Top-performing LLMs, including GPT-4, exhibit low recall in error detection tasks. 2) While LLMs can correctly identify errors, their explanations remain unreliable. 3) The performance of error detection is notably sensitive to subtle variations in prompts. 4) Previous methods aimed at enhancing error detectors have proven ineffective.

Rating

6

Confidence

3

Ethics flag

1

Reasons to accept

1) The main contribution of this paper is to introduce a novel benchmark, ReaLMistake, for error detection in LLM responses. The benchmark contains various, objective and realistic errors made by LLMs, which is valuable for future research in this area. 2) It provides an extensive analysis of the performances of 11 different LLMs, offering valuable insights into the error detection capabilities of popular LLMs. 3) The paper is well-organized, presenting sufficient experiments, ablations, and detailed analysis.

Reasons to reject

The LLMs employed to generate the response (GPT4 and Llama2) could impact the reliability of the benchmark. Another concern is about the sample size. The ReaLMistake benchmark contains a relatively small number of samples compared to other benchmarks with binary labels (e.g., SummEdits and BIG-Bench Mistake).

Questions to authors

1) In Table 2, I notice that the dataset is biased, especially the error percentage of the Llama2 response is super high (75.0, 78.8, 77.5). How different data distribution affects the error detection results? What is the reasonable range of the error percentage? 2) A following question of 1): In Table 4, is it suitable to use your defined "Random" recall score as the baseline? Your dataset with Llama 2 is super biased and to give a more extreme example, if the error percentage is 100%, no model can beat the recall baseline(100). However, many models work better than random guess (recall=50). In this case, I think your claim "LLM-based error detectors often perform worse than random baselines" is a bit unsound.

Reviewer sN459/10 · confidence 5/52024-05-11

Summary

This paper proposes. new benchmark, ReaLMistake, that evaluates how good LLMs are at detecting objective errors. The authors first constructed three sets of instructions (math word problem generation, fine-grained fact verification, and answerability classification) and prompted GPT-4/Llama-2-70B to generate responses. Then the authors manually annotated the errors in the model responses with rationales. The error types are categorized into (1) instruction following, (2) reasoning correctness, (3) context-faithfulness, and (4) parameterized knowledge. All four are common errors of LLMs and can be objectively evaluated. The human agreement rate is high (95.9 F1). Then the authors evaluated a number of open-source/closed-source LLMs on the benchmark -- this includes both the accuracy on detecting the errors and a manual evaluation on the rationales generated by the models. The experiment results are quite interesting: (1) even the best models struggle at accurately predicting the errors. More surprisingly, there seems to be a negative correlation between model capacity (measured by Chatbot arena score) and the error detection recall. (2) The rationales generated by models can be very wrong even if the prediction is correct, especially for open-source models. (3) Prompt engineering is very difficult for error detection and even more advanced prompting techniques like self-consistency does not help.

Rating

9

Confidence

5

Ethics flag

1

Reasons to accept

(1) This is a well-motivated and high-quality dataset that will provide high utility to the language model community. The dataset annotations are manually curated by the authors and have a high human agreement rate. The schemas (source datasets and error types) are intuitive and comprehensive. Not only is error detection an important applications of LLMs, but this dataset (ReaLMistake) can serve as a benchmark for evaluating general LM abilities as well. (2) The experiments are comprehensive and insightful, especially (a) the negative correlation between model capabilities and error detection recall, and (b) the prompt sensitively of the task. Our community will benefit from those insights for better understanding LLM capabilities and building better LLMs/error detection models.

Reasons to reject

(1) The source tasks are limited to synthetic tasks like question generation and fact verification. It would be more interesting to see such a dataset constructed on more diverse instructions such as ShareGPT/Alpaca/OpenAssistant. (2) The negative correlation between the LLM capacity and error detection recall, though interesting, is a bit unintuitive. The authors also did not offer a reasonable guess of why this happens -- which makes me wonder if this is due to suboptimal designs of the prompt.

Questions to authors

If I understand correctly, the annotation includes the types of the error, but the performance on Table 4 and the prompts used by models are simply binary error detection. Would it make the model perform better if they are prompted with the error type definitions and are prompted to provide the error type as well? The experiments would be more insightful as well if a per-type analysis is provided.

Reviewer 3EKz7/10 · confidence 4/52024-05-12

Summary

The paper introduces a benchmark for detecting errors in LLM-generated text. The dataset is generated by modifying several existing datasets (e.g. AQuA, HotpotQA), by prompting an LLM to modify existing questions according to certain instructions. The LLM-generated responses are manually filtered for those that failed to correctly follow the instructions. After constructing this dataset, the paper evaluates a number of LLMs, and finds low recall in identifying errors. The paper also finds that, even when errors are correctly identified, the LLM-generated explanations of these errors are frequently incorrect.

Rating

7

Confidence

4

Ethics flag

1

Reasons to accept

- The paper is addressing an important use case. It would be useful to have LLM-based error detection in many applications. - The method for constructing the errors is scalable and reliable. The errors are manually verified, and there is very high inter-rater agreement. - The experimental results are interesting and convincing. The paper evaluates an appropriate set of models, and finds strikingly poor performance in most cases. - The paper is clearly written.

Reasons to reject

- It would be interesting to understand what factors make an error hard to identify. The results in Table 4 are fairly coarse grained. Does the dataset support more fine-grained evaluations? - This is not very important, but it would be useful to evaluate prompting strategies more comprehensively, beyond what is described in Section 4.3. It seems like it should be possible to improve recall at the cost of lower precision. How large is that cost? - The errors are generated from a limited number of templates on three tasks. It is uncertain how generalizable the results will be to other types of errors, and those that occur in the wild. The paper is nonetheless a good starting point.

Reviewer 9NZk6/10 · confidence 4/52024-05-15

Summary

The paper introduces ReaLMistake, which the authors claim is the first error detection benchmarking suite consisting of objective, realistic and diverse errors made by LLMs. ReaLMistake contains three tasks that introduce objectively assessable errors in four categories (reasoning correctness, instruction-following, context-faithfulness, and parameterised knowledge), eliciting naturally observed and diverse errors in responses of GPT-3 and Llama 270B annotated by experts on three tasks three tasks (Math Word Problem Generation, Fine-grained Fact Verification, and Answerability Classification). The benchmark includes error annotations (binary error label, error categories, and human explanation about errors) on responses from GPT-4 and Llama 2 70B on the three tasks. The authors show that top LLMs like GPT-4 detect errors made LLMs at very low recall, and all LLM-based error detectors perform much worse than humans, that explanations by LLM-based error detectors lack reliability and LLM-based error detection is sensitive to small changes in prompts but remains challenging to improve, and finally, popular approaches to improve LLMs, including self-consistency and majority vote, do not improve the error detection performances. The authors promise to make the code and data available on acceptance. Overall the quality of the paper is quite high in terms of the detail of the description of the dataset, though this is somewhat in part due to the lengthy Appendices making very clear what the prompts and responses were. ReaLMistake undoubtedly could be part of a good resource for evaluation for LLM error detection, though it is limited in scope, as partially admitted to by the authors. The presentation overall is quite strong and clear, though see below for some queries about it which should be addressed. It seems more incremental research than highly novel, as the authors define the benchmarking dataset as an intersection of what has gone before, however the paper claims ReaLMistake is novel in focusing on errors that are unambiguously judged as such by human annotators (though see concerns on this claim below). The approach is quite principled scientifically, though I would dispute the claim of unambiguously annotated error data as being essential for good error detection benchmarking for LLMs, and perhaps if accepted, the authors can reduce the strength of that claim and describe alternatives where multiple annotator’s judgments are included (see below).

Rating

6

Confidence

4

Ethics flag

1

Reasons to accept

There is a useful data contribution of the 900 instances of error annotations by experts, including human explanations for the judgment of an error on responses from GPT-4-0613. This is potentially very useful for certain kinds of error detection benchmarking and is challenging, as shown by the authors who show that near state-of-the-art models such as GPT-4x exhibit a very high error rate in detecting errors in certain tasks with the prompts designed by the authors. The statistics of the ReaLMistake benchmark are clear and show a good range of errors in the different dimensions defined by the authors. The Appendices are very useful as a clear exemplification of prompts and further results, which when included, mean the reproducibility of the paper is quite high from the text alone. Overall the paper is well-written, and fairly easy for readers to understand the method and approach employed.

Reasons to reject

My main reservation about the paper, which could be corrected if claims were slightly toned down, is in terms of the motivation of the form of the benchmark: the authors claim they are against human subjective ratings of errors in a response, or at least instances which are not universally agreed upon for their label (e.g. “We cannot evaluate error detectors if humans cannot determine whether LLM responses include errors”). However, this claim is too strong, and it is not clear how the inclusion of multiple (potentially differing) annotations would make things worse for a benchmark – they are certainly a challenge to be solved going forward, but rigorous inter-annotator agreement studies in creating these benchmark datasets could be a way to deal with ambiguous/non-unanimous labelling - in fact it could even be a good idea to include the differing annotations in the benchmark. The challenge would then shift away from limiting a dataset to unambiguous error instances only to developing systems that give similar confidence levels to the level of inter-annotator agreement. Even in some cases there are mathematics-based prompts, there is still room for ambiguity in the interpretation of verbal prompts. It is not clear whether the explanations were agreed upon between the annotators. It is not made clear how novel the benchmark is as Table 1 suggests ‘MT-Bench, PandaLM, LLMEvalˆ2 Multiple NLP Tasks’ has more criteria and in multiple domains.

Questions to authors

I would reduce the strength of the claims around ambiguity and frame your dataset as collecting error judgements which are low in ambiguity (as illustrated by the 95% F1 score of agreement), rather than ‘objective’. This is scientifically too strong. The claim of it being a diverse set of errors might also be a bit strong, as it is dependent on the diversity and difficulty of the original tasks – I would reduce this and discuss that there is more work to be done to enlarge the scope of error type. As per the above, it is not clear how novel the approach is, given Table 1’s column ticks – please further clarify how it is going further than the top row’s system in the text, particularly if you lessen the claim that human inter-subjective disagreement is a bad thing for annotation overall. Please calculate and report Cohen’s kappa for the agreement score, which is similar to F1 but more comparable to other inter-annotator agreement scores. There are some errors in phrasing throughout which require a proof-read - e.g. - Use ‘LLM-based’ (rather than LLMs-based) - Use determiners in places like ‘We introduce ReaLMistake benchmark’ –> ‘We introduce the ReaLMistake benchmark’

Reviewer 3EKz2024-06-05

Thank you for your response. >We agree that creating more diverse benchmarks is a promising direction for future research. However, collecting unambiguous errors is difficult in tasks in the wild, which are often open-ended and subjective. I am sympathetic to the difficulty of collecting errors in the wild, and agree that this is a good place to start. Nonetheless, it is not clear how well the results will generalize.

Reviewer BGkA2024-06-06

Thank you for answering my questions. I'll keep the score.

Reviewer 9NZk2024-06-07

Response to the rebuttal

This is good the Cohen's kappa figures have been added and would be added to an accepted version. The clarification on pair-wise benchmarks vs pointwise are welcome, and again would be good to include to discuss the limitations. My reservation is that the claims must be toned down about the unambiguous nature of the data - no natural language data is fully unambiguous.

Program Chairsdecision2024-07-10

Decision

Accept

© 2026 NYSGPT2525 LLC