Summary
This paper studies the cause of neural text degeneration, i.e. language models tend to generate repetitive loops. They design an experiment showing that text degeneration is correlated with the amount of repetitive text in the training data. Motivated by this finding, they propose repetitive dropout, which applies dropout on the attention weights over repetitive context. They experiment with 3 datasets and demonstrate that their proposed method can greatly reduce repetition. They also re-inspect previous hypotheses:
1. high in-flow words: they show that merging repetitive words contributes to a large portion of the effect of merging high in-flow words.
2. the maximum likelihood objective: they show that even when the model is trained with the maximum likelihood objective, as long as repetitions are penalized, the model does not degenerate much. Thus, the maximum likelihood objective is not the cause of neural text degeneration.
3. self-reinforcement of repetition: the argue that
a. It is also caused by the repetitive words in the training data.
b. The self-reinforcement loops are broken by their proposed method.
Finally, they categorize repetitions into three categories and study their frequency along with their effect on models’ degeneration behavior. They find that the *theme* category has the greatest effect.
Strengths
1. Most parts of the paper are easy to follow.
2. Neural text degeneration has been discovered since 2019, but its cause is still unclear. Finding the root cause may interest audiences in this field.
3. Despite the hypothesis being simple, as far as I know, it hasn’t been studied. The experiment design in Section 4 is clever.
4. They proposed a simple method that can reduce repetition effectively.
5. They inspected a few previous hypotheses and the results are aligned with their hypothesis (in general).
6. They inspected the effect of three types of repetitions in the training data, which in my opinion is insightful.
In sum, in my opinion, their analyses of the cause of neural text degeneration is comprehensive and the conclusion is convincing.
Weaknesses
In my opinion, because the main message of this paper is clear enough, the following issues may not be very crucial.
1. The authors only compare their method with one baseline in Table 1. There are other mitigations for text degeneration, e.g. [14, 22, 11] cited in the paper.
2. Section 6.2 is relatively hard to follow.
1. Line 248, “The core idea of many previous works is to penalize a specific set of data to alleviate the degeneration.” is vague, though I can roughly figure out what it means after reading the following parts.
2. Having a brief introduction for the definition of high-inflow words will make the paper more self-contained.
3. For the Likelihood Objective part, the paragraph at line 283 needs a clearer topic sentence. The connection between the experiment design in paragraph at line 288 and the argument in the previous paragraph is not very clear to me either.
3. At Line 312, the author mentioned “we find that the model trained by repetition dropout can break the self-reinforcement loop”. Some evidence should be provided.
4. At Line 223, “we hypothesize that the exposure bias issue is an important factor for the left repetition, because we find that lower-order n -gram2…”, I couldn't understand the explanation. Some details should be provided too.
repetitions generally appear earlier than higher-order repetitions
4. Given the popularity of large language models, the study on GPT-2-sized models may be less useful for the general audience.
Questions
1. If repetition in the training data is the cause of degeneration, why not simply remove those repetitive text from the training data? This seems to me a more straightforward method than applying attention dropout.
2. If the authors promise to address the issues in the weakness section above, I would be happy to increase my score, because I think the main message is clear and interesting enough.
3. Having some analyses on large language models would make this work more impactful, e.g.
1. If I understood correctly, the analyses in Section 6.3 can also be done for LLMs.
2. I also wonder what would be an effective way to fine-tune an existing (large) model so it doesn't generate repetitive text.
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Limitations
Limitations are discussed in Section 7.