On the Noise Robustness of In-Context Learning for Text Generation

Summary

The paper introduces a new method to deal with noisy annotations for in-context learning. The authors suppose that the examples that cause higher perplexity than their neighbors are more likely corrupted than their neighbors. So, the authors suggest replacing the examples, causing suspiciously high perplexity by their neighbors with lower perplexity (the process is formalized by the "Local Perplexity Ranking" formula). They show experimentally that this method significantly improves the quality of in-context learning for several tasks for several models of size 13B or less.

Strengths

- The paper approaches an important problem with a novel method that significantly improves the quality of LLMs in in-context learning scenarios; - The main idea of the paper is easy to understand.

Weaknesses

- In Table 2, the authors compare their method with several demonstration selection methods. However, it is still unclear whether **any** demonstration selection method, including LPR, is really the best way to improve the model quality in this situation. What about simpler methods to improve the answer quality, such as a chain of thought? What if it could work better even when coupled with the demonstration of noisy examples? Adding more types of baselines, such as some simple variations of a chain of thoughts and chain of thoughts + noisy examples would make the paper's claims much more solid, but there is no such comparison. - See the "Questions" section and "Limitations" section.

Questions

Essential questions: - Why Perplexity = Inherent Perplexity + Matching Perplexity? I didn't find any experimental or theoretical confirmation. - In Figure 2, there is the perplexity distribution of Llama2-7B on clean and noisy annotations. Does this perplexity distribution change for bigger models, such as Llama-13B? Does it change for smaller models, such as OPT? - How to choose a good gamma? You wrote in lines 233-234 that "LPR shows robustness to the choice of threshold γ", but I didn't find any experimental confirmation of this point. Questions about the presentation: - What model is used for Table 2 results? Is it LLAMA-2 7B? - What metric is shown in Table 3? - You wrote that figures 3 (a)-(d) use different gamma values, but which exactly? There is no information in the figures caption.

Rating

7

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

I would suggest adding the lack of any not-demonstration-selection-based baselines (such as chain of thought) and a narrow scope of models to the limitations section. **Or** I would suggest performing corresponding experiments and adding them to the paper if it is possible to do so in a limited time.

Summary

The paper "On the Noise Robustness of In-Context Learning for Text Generation" investigates how LLMs handle noisy annotations during in-context learning (ICL). The authors propose a method called Local Perplexity Ranking (LPR) that replaces noisy candidates with nearby neighbors that are less noisy. They also explore the impact of noisy labels on ICL performance and compare different selection methods for these examples, such as TopK and DPP. Their proposed method can effectively mitigate the negative effects of noise, offering insights into enhancing the noise robustness of ICL in practical applications.

Strengths

- The paper introduces a novel approach (LPR) to address the issue of noisy annotations in in-context learning for text generation tasks. This is particularly original as it challenges previous assumptions about the robustness of in-context learning to noisy demonstrations in classification tasks. - The paper is generally well-structured and easy to follow. The definitions and metrics are defined properly. The methodology is described in clear detail. - The paper shows that LPR does improve the performance of various selection methods, including Random, TopK, and DPP, especially in the presence of noisy annotations. The proposed method addresses an important and practical issue in the field.

Weaknesses

- The absence of code makes it difficult for the broader research community to reproduce the results claimed in the paper or verify the method's effectiveness on the tasks (the authors claim that the code is released in section 4, but I'm not sure where it is). - The paper did not provide an analysis of the types of errors that LPR makes versus the baseline methods. Such an analysis could provide insights into the strengths and weaknesses of the approach. - While LPR is compared to baseline selection methods like TopK and DPP, it is not compared to other techniques specifically designed to handle noisy annotations during in-context learning.

Questions

- What are the specific cases where LPR may tend to fail? Does each model's inherent capabilities affect these results? - What is the specific model version for GPT-4 and the approximate cost for generating relevant noise for each task? What was the specific method (or prompt) used for generating relevant/irrelevant noise?

Rating

7

Confidence

3

Soundness

3

Presentation

4

Contribution

3

Limitations

The authors have addressed the potential limitations of this paper. It would be useful if the authors also present examples where their proposed method have failed.

Summary

This paper proposes Local Perplexity Ranking (LPR), a method to improve the robustness of in-context learning for text generation tasks when dealing with noisy annotations. The key contributions are: - Empirically demonstrating that noisy annotations hurts performance of in-context learning for text generation, unlike for classification tasks. - Proposing LPR, which ranks candidate demonstrations based on perplexity within local semantic neighborhoods to identify and replace likely noisy examples. - Experiments showing LPR improves noise robustness in most scenarios across multiple text generation datasets, 2 noise types, and 3 baseline selection methods. The method is motivated by analyzing perplexity distributions of clean vs noisy examples and decomposing perplexity into inherent and matching components. Overall, LPR provides a simple but effective approach to mitigate issues with noisy demonstrations in in-context learning for text generation.

Strengths

1. Provides clear analysis on the effect of noisy labels to text generation tasks. 2. The explanations in the disentanglement of perplexity justifies the method of LPR. 3. Comprehensive empirical analysis and ablation studies in the appendix.

Weaknesses

1. The author states that the paper's motivation stems from the occurrence of noisy annotations in in-context demonstrations: > For those candidates, input-label mappings solicited from humans [58, 68] or LLMs [55] can often be noisy, especially in **complex tasks**. This gives rise to the importance of noise-robust ICL, which aims to construct effective demonstrations in the presence of noisy and erroneous labels. However, the current evaluation of the proposed LPR approach is limited to short-form, closed-domain question-answering tasks using traditional NLP datasets. These datasets typically don't suffer from noisy annotations, as the tasks are relatively simple, and ensuring the correctness of a few (fewer than 10) in-context samples shouldn't be challenging. Including experiments on long-form, open-domain question-answering tasks would better justify the paper's motivation and demonstrate the broader applicability of LPR to more novel tasks. 2. The evaluation experiments on justifying the benefits of LPR are conducted using noisy annotations with noise ratios exceeding 20%, which is likely unrealistic for short-form QA tasks in real-world scenarios. This experimental setting appears overly synthetic. 3. It would be helpful if the author could provide a clear illustration demonstrating how LPM is conducted.

Questions

1. Have the authors conducted experiments on long-form, open-domain question-answering tasks (e.g., MT-Bench)? 2. I observe that the benefits of LPR decrease as the noise level diminishes. In fact, in Table 2, when the labels are clean, the effect of LPR appears negligible, as it doesn't improve upon the baseline 50% of the time. While I understand that LPR is designed to handle noisy labels, have the authors conducted experiments with noise ratios between 0% and 20%? At what threshold does LPR start to show a noticeable effect compared to the baseline?

Rating

5

Confidence

3

Soundness

3

Presentation

3

Contribution

2

Limitations

Yes. Would be great to further discuss some possible failure mode of LPR.

Response to the authors

I appreciate the author's responses and the experiments conducted on MT-Bench and Arena-Hard, as well as the additional experiments with small noise ratios. I have raised my score accordingly.

Response to authors

The authors' response resolves most of my issues. Thus, I have changed my score accordingly.