Summary
The paper is focusing on the important open problem of evaluation in natural language generation. Using a dataset consisting of 40 questions of intentionally flawed machine generated (GPT4) answers, the authors employ both human evaluation (crowd-sourced and expert) and LLM-based evaluation focusing on three aspects of the generated texts: i) language proficiency, ii) factual accuracy, and iii) response length. Their analysis reveals biases in the evaluation process: human hesitancy vs LLMs certainty in determining answer quality, humans do not thoroughly fact-check answers vs LLMs displaying some degree of fact-checking ability, and both human and LLM judges favor longer texts. Moreover, factually incorrect models are preferred over those that generate grammatically incorrect or short responses. Informed by these findings, the authors recommend the evaluation of texts from multiple perspectives as opposed to a single metric only, and propose a rating system that assesses the quality of machine-generated texts accounting for i) language, ii) accuracy, and iii) helpfulness criteria.
Strengths
The topic of the paper is very relevant and timely since evaluation is an open problem in the literature. The community needs a thorough understanding of current evaluation metrics and informed by their limitations, better evaluation metrics. The paper makes a step in this direction.
Weaknesses
Their evaluation setup is limited and includes dataset of only 40 questions. The sample size is small and insufficient for drawing convincing conclusions. The authors also do not discuss the importance of the categories of these questions (“generic”, “knowledge”, “common sense”, and “counterfactual”) after leaving out other categories.
The expert evaluation section (2.4) discusses games, but these games are not introduced/presented ahead of this section: “we sample 200 games out of a total pool of 5280 games, which is statistically sufficient to yield meaningful in- sights into the model’s performance”.
“By incorporating multiple LLM judges, we can establish a more comprehensive and unbiased assessment of the generated answers in our study.” - Generally agree with this statement, however in the experiments only GPT-4 and Claude (Antrophic) are used, how is this in line with your claims? Only two LLM-based models is not reflective of the goals of your statement. In addition, Claude is not even mentioned in the Introduction or anytime before Section 2.5, leaving the reader the impression you are only using GPT4.
The findings of the user study conducted by the authors are unsurprising. The length of the text is a well-known confounder in the evaluation of generated texts. Similarly, texts which contain grammatical errors can be quickly identified as having poor quality. Moreover, it is unsurprising that humans tend to be biased towards factually incorrect answers that look coherent and are grammatically correct (especially true if they lack in-domain knowledge). Besides, the authors make misleading statements when claiming LLMs reach consensus when in fact GPT-4 and Claude only reach moderate agreement, and that LLMs display some degree of fact checking abilities - hard to quantify this statement, since the lack of factuality is an open problem with current models. Similarly, the authors claim “the expert annotators outperform the crowd-sourced annotators in the evaluation, despite not being entirely error-free themselves.” - this is a well known fact in the literature, however it is presented as if it were a novel insight which is very misleading!
“expert annotators indeed prioritize factual accuracy during evaluation.” - of course, since they have in-domain knowledge. However, there is no discussion highlighting the relevance of the selected experts vs the selection of the specific domains the authors choose to focus on in this work.
“Regarding the “Helpfulness” dimension, both expert annotators and GPT-4 consistently consider longer answers to be more helpful.” - of course, since they contain more information! Nevertheless, the authors only make vague assumptions and fail to quantify the degree of factuality to support their statements.
The setup is not consistent: it evaluates i) language proficiency, ii) factual accuracy, and iii) response length, and then proposes a rating system that accounts for i) language, ii) accuracy, and iii) helpfulness. Helpfulness criteria is not clearly motivated, and its inclusion is inconsistent with the initial evaluation criteria. Besides, the paper does not clearly describe how this aspect is being evaluated, to whom it is helpful and in which contexts.
There is no clear motivation in the paper on why the authors chose to focus on these particular three aspects ( language proficiency, factual accuracy, and response length) of machine-generated texts. There are so many aspects of the generated texts that can be considered, and this paper fails to make a convincing argument on the selection of their evaluation criteria. While the authors acknowledge in the Discussion section that this may not be the perfect setup, they fail to motivate their experimental choice.
The paper fails to discuss limitations of human evaluation, as well as limitations of LLM-based evaluation with respect to findings already well-established in the literature. Their statements in the Discussion section are vague and unsupported by evidence.
The Related Work section is not very informative for the topic of the paper, particularly the paragraph focused on enumerating existing LLMs is not discussing evaluation at all (which is the main focus of the current paper). In addition, the authors fail to cite and refer to important bodies of work in the literature focused on both human and machine evaluation. I recommend a complete rewrite of this section and instead of presenting generalities, discuss papers that have addressed the same topic and are particularly relevant to your current work.
In general, the experiment setup decisions made in this paper seem rather arbitrary and not well documented / supported by the arguments presented. The authors fail to discuss and include significant papers focused on the evaluation of natural language generation (NLG) models, and do not place their work in the context of recent literature on NLG evaluation.