Long-form factuality in large language models

Summary

This paper introduces LongFact, a dataset consisting of 2,280 questions across 38 topics, designed to assess the factuality of long-form answers generated by large language models. The authors also propose a new evaluation method called the Search-Augmented Factuality Evaluator (SAFE), which utilizes large language models and Google search to verify the accuracy of individual facts within long responses. Additionally, the paper introduces the F1@K metric, which is used to measure the precision and recall of factual accuracy in the model's responses.

Strengths

1. The workload of the paper is substantial, including the construction of the dataset, evaluation methods, and assessment metrics. 2. The paper introduces the comprehensive LongFact dataset, which covers 38 topics, providing broad domain coverage that enhances the depth and breadth of the evaluation. 3. The SAFE method reduces the cost and time of manual evaluation.

Weaknesses

1. The layout of this paper could be improved. Many significant pieces of information have been relegated to the appendix, which may result in key details and data being overlooked by readers at first glance. For instance, the SAEE method, one of the contributions of this paper, should be prominently discussed in the main body of the text. However, it is only briefly mentioned in passing, which does not justify its importance. 2. The paper frequently cites the FactScore method as "such as Min et al. (2023)" without specifying which aspects were used. The author should at least summarize the key elements adopted, rather than requiring readers to refer to the original paper for details. 3. Table 16 shows repeated queries for each fact, which contradicts the claim in the "Rating Individual Facts" section that including retrieved results in prompts prevents duplicates. The author needs to address this inconsistency. 4.

Questions

The author mentions that SAFE corrected 76% of the discrepancies with human annotations. Does this suggest that the annotation results from Min et al. (2023) are flawed? If so, does the reported 72% agreement rate with data annotated by Min et al. (2023) still hold significance?

Rating

7

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

The results in Table 2 should be differentiated, such as by bolding the best outcomes.

Summary

In this work, the authors investigate the evaluation of LLMs’ factuality in long-form generation. Specifically, they first introduce LongFact, a multi-topic benchmark for assessing long-form factuality. In LongFact, they use GPT-4 to generate questions about specific concepts or objects from given topics. Additionally, they design a search-augmented factuality evaluator (SAFE), which evaluates the LM’s output by breaking down the generation into individual claims and then using the Google Search API to verify the correctness of these claims. They also conduct experiments with popular LLMs and find that larger models exhibit better factuality in long-form generation.

Strengths

Originality: The authors propose a new benchmark for evaluating long-form generation. Unlike previous studies, LongFact encompasses multiple topics. The SAFE evaluation pipeline introduces the use of the Google Search API to estimate the correctness of claims, a novel approach not previously employed. Quality: This paper explores both the effectiveness and the cost of evaluation compared to human annotators, providing strong support for the value of the proposed methods. The error analysis offers valuable insights into their method and helps the reader better understand the evaluation pipeline. Clarity: The paper is well-structured and easy to follow, with detailed explanations provided. Significance: This paper tackles the challenging task of evaluating LLM factuality in long-form generation. The datasets are released, and the results are easily reproducible, benefiting the research community.

Weaknesses

1. The overall concept is similar to previous work that decomposes long-form responses into claims, such as FactScore, which diminishes the novelty. Although this work covers more topics and uses Google instead of Wikipedia in the evaluation pipeline, the similarity remains. 2. The claim that “LLM agents can be better factuality annotators than humans” seems somewhat exaggerated. The human annotators do not have internet access and could only label based on the corresponding Wikipedia page, which is not a practical setup.

Questions

1. I notice that current benchmarks for long-form factuality, including LongFact and others, typically consist of questions that ask for information or explanations about a subject, such as individuals or concepts (typically extracted from Wikipedia). The dependencies between sentences in the LM’s responses in these benchmarks are generally simple. Can SAFE handle responses where more complex inter-sentence dependencies exist? Consider the following question from ELI5 as an example: Query Title: What were different decades in the 1800s like? Explanation: Every decade of the 20th century can be loosely quantified in a general theme (the roaring 20s, the flower power 60s, the weird clothing styles/music of the 80s) and each one is so different from the last. What kinds of themes were common between decades in the 19th century, and were they as distinctively different from each other as those in the 20th century? Candidate Answer: In the early 1800´s there was a radical change of fashion and lifestyle for men. The high breeches, stockings and tricorne hats one associates with the era of the French and American revolutions disappeared with the genesis of modern fashion which emphasised modesty in clothing, eloquence in speech and the most refined manners and hygiene. This was the birth of the "dandy" culture which still thrives and is very much predominant today. In this candidate answer, the claim “This was the birth of the ‘dandy’ culture” is supported by all the previous claims. Could SAFE handle this? 2. It seems that the idea of constructing LongFact and the quality estimation method SAFE could be used to produce synthetic data for enhancing LM’s factuality. Do the authors plan to explore this direction?

Rating

8

Confidence

3

Soundness

4

Presentation

4

Contribution

4

Limitations

The authors effectively summarize the limitations of this work. Most of these limitations are not trivial to address, and I agree with the authors that the current design represents a good trade-off between cost and performance.

Summary

This paper focused on the open-domain long-form factuality problems of Large Language models. It proposes a benchmark called LongFact, which consists of more than 2k prompts across 38 domains, it also proposes an agent-based factuality detection system called SAFE with a new metric to measure the factuality: F1@K. Experiments show that SAFE has satisfactory factual error detection ability which surpasses crowd-source annotators.

Strengths

This paper addresses a critical problem and offers substantial contributions. I believe the released benchmark alongside the SAFE system could be practically useful in real-world applications.

Weaknesses

It is foreseeable that the performance of SAFE could surpass that of crowd-sourced annotators, given that crowd-sourced annotations often lack high quality. I think it would be better to compare the SAFE with expert annotators.

Questions

Refer to the weakness.

Rating

7

Confidence

4

Soundness

4

Presentation

4

Contribution

3

Limitations

Refer to the weakness.

Summary

This paper introduces a novel benchmark, LongFact, for evaluating the factual accuracy of long-form responses generated by large language models (LLMs). It proposes a method named SAFE (Search-Augmented Factuality Evaluator) to automatically assess the factuality of these responses. SAFE uses an LLM to decompose a long-form response into individual facts and checks each fact's accuracy through Google Search. The authors introduce F1@K, a metric that balances the precision and recall of supported facts to quantify long-form factuality. Empirical results show that SAFE outperforms human annotators in terms of cost and accuracy. The study benchmarks thirteen language models across four model families, finding that larger models generally produce more factual responses.

Strengths

- Innovative Benchmark: The creation of LongFact, a comprehensive benchmark specifically designed for long-form factuality, addresses a significant gap in existing evaluation methods that primarily focus on short-form responses. - Automated Evaluation Method: The development of SAFE, which leverages LLMs and Google Search for fact-checking, provides a scalable and cost-effective alternative to human annotations, significantly reducing evaluation costs while maintaining high accuracy. - New Metric: The introduction of F1@K as a metric to balance precision and recall in evaluating long-form factuality is a notable contribution, offering a more nuanced measure of a model's performance.

Weaknesses

- Reliance on Google Search: The dependence on Google Search as the primary source for fact-checking may introduce biases and limitations, The retrieved information may also contain inaccuracies or hallucinations, especially for topics that are not well-covered or are subject to misinformation online. - Limited Comparison of LLMs: The paper would benefit from a more comprehensive comparison of human agreement scores using different LLMs within the SAFE framework, including open-source models like LLaMA2. This would provide a clearer evaluation of the factuality assessment capabilities of various LLMs. - Insufficient Ablation Studies: Conducting more ablation studies would strengthen the paper, such as comparing human agreement scores using different search engines or searching across various sources. This would help understand how different fact-checking methods impact the accuracy and reliability of the evaluations. - Generalization to Other Domains: The paper focuses primarily on open-domain factuality. It remains unclear how well the proposed methods generalize to specialized domains such as medicine or law, where factual accuracy is critical and more nuanced.

Questions

Could you please provide more details of re-annotation on the randomly-sampled subset of 100 individual facts?

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

Yes

Summary

This work proposes a 4-step pipeline for automatic evaluation of long-form answers using LLMs. Given a long-form answer, their pipeline involves (1) splitting the question into individual facts, (2) decontextualizing each fact, (3) determine each facts relevance to the original question, and (4) verifying each fact given retrieval results using Google search. In addition to proposing this pipeline, the authors also develop a set of 2,280 total information seeking prompts (automatically generated w/ GPT-4) for evaluation. The authors then demonstrate the efficacy of their evaluation pipeline by comparing their pipeline's judgements against human-annotators and find high (~75%) agreement. Furthermore, the authors find that, in these disagreements, their pipelines judgments were usually correct and the annotator judgments were flawed (also ~75% of the time). Finally, using their pipeline, the authors propose a metric for evaluating the quality of a long-form response that is based on F1 over individual facts in the generated answer. Instead of relying on a gold set of individual facts for each question, their metric evaluated recall using an "ideal number of individual facts in the answer for a given question" which is set as a hyperparameter of their evaluation metric.

Strengths

This work address an important open problem in LLM evaluation, automatic evaluation of long-form answers. The proposed evaluation pipeline, is accurate (as demonstrated through human evaluation) and cheap (20x cheaper than relying on humans). The proposed dataset as well, will be a great resource for future work. While the proposed F1 evaluation metric may not comprehensively evaluate all aspects of individual fact recall in long-form QA, it still represents a suitable and scalable alternative. See Weaknesses below for discussion.

Weaknesses

One concern is with the "ideal number of individual facts in the answer for a given question" hyperparemeter. While this does to be a suitable alternative to relying on a gold set of facts, such a value is highly question and user dependent. While the results demonstrate that performance rankings are robust to this selection in hyperparameter, there is some concern about whether models that have been trained to produce more concise answers versus more verbose answers. Intuitively, it seems that fact relevance (not just the binary prediction, but some more granular score) is also an important consideration when evaluating recall. Recall over a wide range of correct, somewhat relevant facts may be less desirable than recall over a small number of highly relevant facts.

Questions

Regarding weaknesses noted above, I'm curious whether, if this evaluation / metric / dataset is released as a benchmark or leaderboard, if it could be gamified by systems that are trained or prompted to produce a specific number of relevant facts (matching K in the evaluation metrics)? It seems like something along the lines of the precision vs # of facts recalled curve in the appendix seems to be a more suitable way to evaluate models, testing their ability to generate responses with high F1 over different long-form answer lengths.

Rating

8

Confidence

4

Soundness

3

Presentation

4

Contribution

4

Limitations

Yes

Further questions

Thanks for the author's response. I still have some questions. 1. When the author used SAFE for evaluation, they employed Google search. For each query, how many search results were used to conduct fact verification? 2. Although the author states that large language models do not always follow human instructions, looking at the sample tables 16-18 provided by the author, there are too many repeated queries. I'm somewhat concerned whether your query generation strategy is truly effective. 3. When the author used an LLM agent for fact annotation, they re-annotated the results where SAFE and human raters disagreed by using Google search. What was the purpose of this? Was it because the annotations of these inconsistent results were incorrect? The author denies this. Can you explain this?