Towards Verifiable Text Generation with Symbolic References

Summary

The authors propose a new prompting technique to generate textual descriptions grounded in structured data, e.g., a news blurb describing financial data or sports statistics. The key innovation appears to be the use of Jinja-like syntax to inline symbol references in the generated text. The authors evaluate this approach on several data sources: biographies from WikiData, sports data from Rotowire and financial data from a financial API, where they observe that their proposed approach generates descriptions that are comparably fluent and grounded to a baseline approach. The key benefit however are the references which can be used to show readers links to the source data: through a user study, the authors demonstrate that these links allow readers to verify the data presented in the text more quickly and easily. Finally, the authors show two other ways in which the symbolic references could be useful: first to help language models override their priors with counterfactual data, and second as a way to help language models use symbolic expressions for math problems. Unfortunately, in both of these applications, the results are mixed and are not consistently better than the baselines.

Rating

6

Confidence

4

Ethics flag

1

Reasons to accept

- The paper is well-written, and is comprehensive, with quantitative and human evaluations, some qualitative error analysis and detailed information on the prompts used. - The main contribution of this paper appears to be showing that their prompt strategy works in providing references to structured data. The authors convincingly show that this makes it easier for users to verify the generated text through a user study, though they do not show that it significantly improves the groundedness of the generations.

Reasons to reject

There are several dimensions along which this paper makes contributions, but unfortunately, I found it hard to find one in which it excels: - Application: The chief contribution of this paper appears to be generating symbolic references along with text to help show those references to users. While valuable, the application has been proposed several times in the literature, and this paper limits its scope to referencing (near-)exact matches from the source data. I think this too narrow, given the state of language models today (e.g., the GPT3.5/4 models used in this paper). - Technique: While the paper provides a clever way to prompt language models to generate text with grounded symbolic references, this technique isn't particularly effective at improving the groundedness of the generations over baselines (which are already very good). Furthermore, given the relatively small evaluation set sizes, it is unclear if any gains are statistically significant or not. - Data: the paper does generate its own synthetic counterfactual obituaries dataset, but the authors do not really analyze this dataset or make it public.

Questions to authors

- In the discussion of the counterfactual evaluation, you mention that some of the difference in performance might be explained as a precision-recall trade off. This seems easily quantifiable, have you measured precision/recall of the generated answers? - It is great that you included dataset sizes in the appendix, but I think this information belongs in the main paper. It is really hard to understand what's a meaningful improvement without that information. Given the relatively small sizes of the evaluation datasets, could you do a statistical analysis and report which results are actually statistically significant?

Summary

The paper presents a novel approach, symbolically grounded generation (SymGen), which enhances the capabilities of large language models (LLMs) for complex reasoning tasks. The authors demonstrate that SymGen can produce text with accurate symbolic references while maintaining fluency and factuality, which is a significant advancement in the field of natural language processing. Please note that I doubt whether the font used in this paper aligns with the Call for Papers (CFP) guidelines for COLM.

Rating

4

Confidence

4

Ethics flag

1

Reasons to accept

1. Innovative Approach: The paper introduces a new method for text generation that significantly improves verifiability without compromising the quality of the text generated by LLMs. 2. Empirical Evidence: Strong empirical results support the claims made by the authors, with improvements demonstrated across a range of data-to-text and question-answering experiments. 3. Practical Application: The proposed SymGen method has practical implications for applications such as virtual assistants and knowledge bases, where the ability to generate verifiable text is crucial. 4. Human Study: The inclusion of a human study adds weight to the findings, showing that the annotations provided by SymGen can streamline the verification process. 5. Well-Structured Paper: The paper adheres to the COLM reviewing guidelines and is presented in a clear, accessible manner, making it a valuable contribution to the literature.

Reasons to reject

1. Limited Dataset Scope and Insignificant Superiority: The evaluation is based on a small number of toy datasets, which may not fully capture the diversity of real-world applications. I expect to see the performance of the method on some more realistic and complex datasets, e.g., table reasoning, where the method may be superior. 2. Complexity of Indirect SymGen: The indirect SymGen approach, while maintaining text quality, could be more costly and less reliable with certain LLMs, which might be a concern for practical adoption. 3. Error Analysis: The paper could benefit from a more detailed error analysis, particularly in understanding the types and causes of errors in symbolic generation. 4. Generalizability: It is unclear how well the proposed method would generalize to other types of structured data (e.g., Graph, Table, Databases, etc.) and language models, which is a key consideration for its broader acceptance. 5. More than Verifiability: The authors emphasize the proposed method's verifiability. But what are the benefits? I am curious about future work. Does better verifiability make it easier for the LLMs to receive human feedback and evolve? Do the authors have more in-depth insights? 6. No Training: The paper does not attempt to train LLMs to master the technique of generating symbolic references, which limits its impact.

Questions to authors

Please see Reasons to Reject

Summary

The authors propose SymGen which provides symbolic references to the input structure (JSON) along with the generation. Two variants are proposed, direct and indirect. The direct approach prompts LLM to generate text along with Jinja-like references in one go. The in-direct approach first prompts LLM to generate a text summary, that is subsequently transformed into text with symbolic references by prompting. The experiments: 1. study changes in textual quality due to symbolic reference generation 2. show that the annotations cannot be simply replaced by regex-based replacement. 3. show the system improves the verifiability speed. 4. show that SymGen can also be applied to QA on structured (JSON) data and reasoning setup.

Rating

6

Confidence

5

Ethics flag

1

Reasons to accept

I believe that such a system is a natural attempt towards this problem. That could mean that the learnings are benificial to the community. I also appreciate that the authors have conducted multiple experiments that answer a variety of questions.

Reasons to reject

In general, there is no clear recommendation by the authors. I see many mixed results, which is fine, but the paper does not map it to clear insights or error analysis. Table 1 (a) and (b) have mixed results depending on the dataset. The paper claims: 1. That rotowire is a difficult dataset and requires a lot of exact numbers replicated. However, the baseline results do not support this. 2. SynthBio has a simpler JSON compared to Rotowire, but the authors suggest that biographical generation in symbolic space is slightly harder. It would be very helpful to have some error analysis to gain an understanding of inherent difficulty. The authors claim that the indirect strategy does not work for GPT3.5 as it is weaker than GPT-4. Due to a lack of examples/comments, it is difficult to say why is that the case. It could be helpful if there were different and possibly open-source models used as well to be able to helpfully distill when this method works. Although I do acknowledge the GPU requirements issue here. There are no details on how the parsing and representation would work for representation other than JSON. Hence, it seems that the proposed method is limited to JSONs. Furthermore, there is a limited description of the parsing stage itself apart from seemingly simple regex + replacement. It is not obvious from the writeup why the authors chose not to use the in-direct method for the experiments in Table 2. For the QA experiments, the gap between GPT-3.5 baseline vs direct methods is huge compared to GPT4, and experiments in Table 1. My assumption is that it could be due to shorter outputs. But the error rate is missing in this experiment which makes the results unclear. In the related work section authors talk about LLM-based verification methods, without justifying the differences or why comparisons cannot be made. On a related note, and agreeing with the associated post-processing cost of prompting, regex-based baseline is not a fair comparison. A prompt-based baseline would be more adequate. An approach could be (only to cite possibilities): for each summary sentence, prompt which key-value pairs support the sentence? Or a prompt: where all sentences are passed as a list at once to make it comparable to indirect generation.

Questions to authors

In section 3.1 : Why does it say "incomplete JSONS"?

Thank you for the response

Thank you for the response and clarifications. Given the author's reply, it appears that not all concerns about the write-up can be resolved without altering the text significantly; therefore, I will be keeping the scores unchanged.

Thank you for your response

Thanks for your detailed response and for clarifying that the scope of the contributions. My concerns mostly related to whether the methods proposed here could improve the factuality of responses, which might be less relevant for the LM and interactions track. I find the prompting method to be quite novel and the user study convincing. I have increased my score to marginally above the acceptance threshold.

Overall comment and planned paper updates

We would like to thank all reviewers for their time, effort, and constructive feedback! We are encouraged that both reviewer XpW4 and oNbF acknowledge the contribution of our system for providing references in generation and reducing the effort for verification, find our experiments comprehensive and results convincing, and recognize the value and benefits of our methods to the community. Most of the reviewers' concern centered around whether SymGen can lead to better text quality: we’ve addressed the concerns in the comments for each reviewer individually. But we want to flag that our work centers on the topic of **LMs and interactions**, and our paper is scoped to tackle the pressing problem of human verification of LLM generations, and both reviewers XpW4 and oNbF appreciate the method we propose and the multiple experiments we conduct to validate the method. We will update our paper following their suggestions. Here is the list of the planned updates based on reviewer feedback: - Highlight our paper’s key contribution for reducing users’ effort for verifying the outputs. (XpW4, oNbF) - Describe the parsing pipeline in depth and mention the opportunities for using other formats (e.g., yaml). (XpW4) - Clarify why we use specific strategy is adopted in experiments in table 1 and 2, and additional description and examples for the Rotowire and Synthbio dataset. (XpW4) - Update the discussion on “precision/recall of generated answers in counterfactual eval“ (oNbF) - Report statistic significance for table 7 (https://imgur.com/a/5IWhJbn, oNbF) - Update the related work section to reflect that our work is among the first to focus on how LLM generations can be made easier to verify by users in data-to-text generation settings. (XpW4, oNbF) We find the suggested changes for the camera-ready version are not substantial (mostly clarifications), and we can incorporate the changes in the final draft within this review cycle.