ContextCite: Attributing Model Generation to Context

Summary

This paper proposes a simple and effective approach for attributing LLM generation to the sources in the context, where a set of random ablations and their corresponding effects in the model's probability of generating the response are modeled with a (sparse) linear relationship. The method archives impressive performance and also allows applications such as verifying model generations or denoising the sources in the context.

Strengths

- The problem of attribution is very important, and the paper focuses on an understudied aspect of how different sources *cause* the model to generate what it generates. - The proposed method is simple and intuitive. Comparison with the baselines is thorough and clearly demonstrates its superior effectiveness. - The paper is generally well-written and easy to follow.

Weaknesses

- The method may be a bit costly to run, given that the linear relationship needs to be re-learned for every (context, generation) instance, and the number of sample ablations also grows when the context scales. - If I'm not mistaken, the proposed linear relationship implicitly assumes that the sources in the context contribute in a rather independent manner to the model's generation. This may not work well when the generation involves complex interactions and reasoning over a large number of sources in the context (beyond just a few sentences as in, e.g., HotpotQA). For example, if the task involves some kind of logical deduction, missing a certain premise in the context could cause a snow-balling effect on how the model processes the remaining sources and scores its generation, which is very hard to model with a linear relation.

Questions

For extracting query-relevant information from the context (Section 5.2), if some important sources are "lost in the middle" and overlooked by the model, then the proposed approach would give a very low score for them as they don't contribute to model generation. Then they also won't get selected during the extraction phase. How would this improve QA performance then?

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

Yes.

Summary

The current paper introduces the task of context attribution, which aims at attributing a generated response of an auto-regressive LM back to the sentences in the input contexts. Target at this task, ContextCite, was proposed to predict which piece of context changes the probability of the generated response most. Together with ContextCite, this paper also designed two evaluation metrics based on the probabilities of the generated responses. At length, the experiments on two small LLMs show the effectiveness of ContextCite. Additionally, this paper provides several examples of applications of ContextCite.

Strengths

- This paper introduces the task of context attribution, which, AFAIK, is the first time this important and interesting task has been introduced. The task is well-motivated and well-defined. One could expect that this paper could initiate a new line of work on interpretability. - Along with the proposed task, this paper proposed a solution, namely ContextCite, building on a simple but effective idea. The subsequent evaluation suggested its promising performance. - One thing that I like the most about this work is that in addition to the task and the model, this paper also introduces two well-designed evaluation metrics as well as a list of potential applications of ContextCite.

Weaknesses

Though I thought this paper was already of good quality, I still have three major concerns regarding the task, the model and the evaluation. First and foremost, in my opinion, the current definition of the task is still highly limited and can be largely extended. For example, the current task only focuses on the attributions with responses to the whole generated responses, but for tasks like summarisation (which was also examined in this work), it is also important to attribute each token in the response to, for instance, route the hallucination. I am personally very OK if the solution of such an enhanced task is not provided, but I am very inclined to see that the related discussions about the possible extensions of the task can be included. Second, regarding ContextCite, though the idea of making use of the probability change looks promising, after due consideration, I have a strong feeling that its relation with which parts in the input contribute more to the response is not fully deterministic. In my opinion, the probability of a response could be reduced more if, as pointed out in the paper, the omitted piece contributes more to the response or only if the resulting context after removing the sentence is way more incoherent. One example of such a possibility is given in Appendix C2, but the discussion seems not to be sufficient. This also made me think that the model only suits tasks whose output relies on the very long input text (e.g., summarisation and document-based QA, as omitting one sentence would not highly influence the coherency). Finally, the evaluation was done on three very related tasks and two tiny language models. This makes me somewhat question whether the solution is generalizable to other NLP tasks (that are open-ended, for example), other larger models and other prompt designs (e.g., in-context learning and chain-of-thought).

Questions

Can you provide some examples of the selected sentences and hit on the distributions and characteristics of the selected sentences?

Rating

7

Confidence

3

Soundness

2

Presentation

3

Contribution

3

Limitations

See my points in the weakness section.

Summary

In an attempt to understand how language models leverage context information in its generation, this work studies contributive context attribution. Following the recent trend in attribution research, authors propose to use (sparse) linear models to learn the importance of each unit/sentence in the context, and demonstrate its effectiveness using top-k log-probability drop and linear datamodeling score (LDS) tests. Lastly, authors explore two potential applications of contributive context attribution, namely generated statement verification and query-relevant information selection.

Strengths

The paper is very clearly written, well-structured, and easy to follow: - The problem of contributive context attribution is relatively new. However, the authors provided all the necessary contexts for the problem, and thus I was able to understand the problem setup easily. - ContextCite seems to achieve promising attribution accuracy on both top-k log-probability drop and LDS tests, especially in comparison to baseline methods. - Their context selection experiment (section 5.2) results also look promising. If we make the analogy between data attribution and context attribution, this probably mirrors a data selection experiment. It's interesting to see that context selection generally leads to the performance improvement.

Weaknesses

- While (contributive) context attribution looks to be new on the surface, their problem setup, method (i.e. ContextCite), and evaluation are mostly borrowed from existing data attribution literature. While authors provided proper references to those literature, I still believe the technical contribution largely lacks novelty. I wasn't able to grasp whether authors introduced any major modifications to adapt existing data attribution methods (i.e. datamodel) and evaluations (i.e. brittleness and LDS) to context attribution. If I missed something, please correct me. - If my understanding is correct, they prompted GPT-4 to generate the (proxy) gold label for their generated statement verification experiments (section 5.1). This was confusing because the authors noted in line 84-85 that prompting the model for citations is generally considered as corroborative attribution. To be precise, the authors asked for the correctness of each sentence instead of for citations here, but it was a bit odd for me to use the prompting technique to evaluate contributive context attribution.

Questions

- How does the context length affect the number of required forward passes with randomly sampled context information? Naively thinking, if you have n parameters in your linear model, you may need n samples to properly fit the linear model (without a sparsity assumption).

Rating

5

Confidence

4

Soundness

3

Presentation

4

Contribution

2

Limitations

The authors adequately addressed the limitations in Appendix C.

Summary

The authors of 17808 formalizes the problem of context attribution for language models. That is, identifying which parts of the input context caused a model to generate a particular output. The authors propose ContextCite, a method that learns a sparse linear surrogate model to approximate how ablating different parts of the context can impact the model's output probability. Authors claim several contributions, including 1) Formalizing the official task of context attribution and evaluation metrics, 2) Developing ContextCite, a simple and scalable attribution method, 3) Demonstrating ContextCite outperforms baselines on attribution quality, and 4) Showing applications in verifying generated statements and improving response quality.

Strengths

- Important problem formulation: Context attribution addresses a key challenge in understanding and improving llm behavior / use cases esp. faithfulness and interpretability. - Comprehensive experiments: Right metric, datasets, and models. Comprehensive experiments are provided. - Case study -- Two valuable downstream applications are addressed - verifying generated statements (5.1) and improving response quality (5.2). - Clear writing and presentation -- Overall I find the presentation of the paper is well-structured and easy-to-follow.

Weaknesses

- As noted in Appendix C.2, the current ablation approach of removing sentences can break dependencies between sents. This limitation could be explored a bit further in the paper, but it's understandable if there's time constraint. - A little more theoretical justification for why the linear surrogate model works well would be better.

Questions

- While the efficiency and simplicity of linear surrogate models are acknowledged, did you try more sophisticated surrogate models beyond linear regression in your early exps? If so, how did they look like?

Rating

8

Confidence

3

Soundness

3

Presentation

4

Contribution

3

Limitations

The authors provide a good discussion of limitations in Appendix C.2.

Missing bibliography

Hi there, While reading this paper, I noticed that the bibliography section is missing. I found the same issue on the version hosted on the NeurIPS website. I hope this helps! Best, Pietro