The Transient Nature of Emergent In-Context Learning in Transformers

Transformer neural networks can exhibit a surprising capacity for in-context learning (ICL) despite not being explicitly trained for it. Prior work has provided a deeper understanding of how ICL emerges in transformers, e.g. through the lens of mechanistic interpretability, Bayesian inference, or by examining the distributional properties of training data. However, in each of these cases, ICL is treated largely as a persistent phenomenon; namely, once ICL emerges, it is assumed to persist asymptotically. Here, we show that the emergence of ICL during transformer training is, in fact, often transient. We train transformers on synthetic data designed so that both ICL and in-weights learning (IWL) strategies can lead to correct predictions. We find that ICL first emerges, then disappears and gives way to IWL, all while the training loss decreases, indicating an asymptotic preference for IWL. The transient nature of ICL is observed in transformers across a range of model sizes and datasets, raising the question of how much to "overtrain" transformers when seeking compact, cheaper-to-run models. We find that L2 regularization may offer a path to more persistent ICL that removes the need for early stopping based on ICL-style validation tasks. Finally, we present initial evidence that ICL transience may be caused by competition between ICL and IWL circuits.

Paper

References (41)

Scroll for more · 29 remaining

Similar papers

Peer review

Reviewer 8T187/10 · confidence 4/52023-07-03

Summary

The paper explores the in-context learning (ICL) phenomenon in transformer networks and investigates its transience during training. It presents empirical findings that ICL if emerged in transformers, may not persist as the model continues to be trained. The paper highlights the importance of evaluating appropriate validation metrics and suggests stopping training before a computing budget is reached to preserve ICL. Strategies to mitigate ICL transience, such as data distributional properties and regularization, are explored. The paper also discusses the competition between ICL and in-weight learning (IWL) circuits and raises questions about the emergence and fading of ICL. The experiments demonstrate that ICL in transformers is often transient. Model size, dataset size, and data distribution can influence the transience of ICL, but persistent ICL is not guaranteed. Wider models with larger embedding sizes show improved ICL, possibly due to reduced competition with IWL circuits. Skewed data distributions mitigate ICL transience but necessitate longer training. Future directions, limitations, and implications for large language models are discussed. Overall, the paper provides valuable insights into the nature of ICL and its implications for transformer networks, contributing to the understanding of in-context learning and its behavior during training.

Strengths

Originality and Contribution: - Empirical findings on the transience of in-context learning (ICL): The paper presents novel empirical evidence that ICL if emerged in transformer networks, may not persist as the model continues to be trained. This finding sheds light on an important characteristic of transformer models and highlights the need for appropriate evaluation and validation metrics. - It provides insights into how ICL emerges, disappears, and gives way to in-weight learning (IWL) during training. - Discussion of the coexistence of ICL and in-weights learning (IWL): The paper explores the competition between ICL and IWL circuits for resources in the transformer residual stream. It raises intriguing questions about the emergence and fading of ICL and offers potential explanations for the coexistence of ICL and IWL in large language models. - Exploration of strategies to mitigate ICL transience: The paper investigates various strategies to mitigate the transience of ICL, including increasing the number of classes, introducing Zipfian data distribution, and using L2 regularization. These strategies provide valuable insights into potential methods for preserving and enhancing ICL capabilities. - It is interesting that ICL is transient even under conditions that are deemed conducive to the emergence of ICL. - Comprehensive and thought-provoking discussion, posing clear and relevant questions: discussion of future research directions and acknowledgment of the limitations of the conducted experiments. It identifies the need for experiments on large language models, exploration of other datasets and context lengths, and investigation of additional mechanisms in transformers, encouraging further advancements in the field. - Consideration of the impact of training duration and stopping criteria: The paper emphasizes the importance of appropriate training duration and stopping criteria to preserving ICL. It suggests that training large transformer models beyond a certain point may result in the gradual disappearance of ICL, even if training losses continue to decrease. This insight has practical implications for model development and training practices. Research Problem and Motivation: - The research problem is well-defined and clearly stated, focusing on the transience of ICL in transformer networks. - The paper provides a convincing motivation for the chosen problem by highlighting the importance of evaluating appropriate validation metrics, preserving ICL, and understanding the factors influencing its persistence. - The relevance and importance of the research problem are clearly explained, emphasizing the potential impact on training large-scale language models. - The paper raises numerous additional questions, stimulating further scientific inquiry and exploration. Methodology: - The methodology is clearly described and well-founded. The paper outlines the experimental setup, including the models, training procedures, and evaluation metrics used. - The algorithms, models, and techniques employed are appropriate for addressing the research problem of studying ICL transience in transformer networks. - The paper adequately explains the experimental setups and data collection processes, providing sufficient details for reproducibility. Experimental Evaluation: - The experimental results are presented clearly and concisely, with appropriate figures and tables. - The evaluation methodology is rigorous and appropriate, considering various metrics and comparing experimental conditions. - While the statistical significance of the experimental results is not explicitly mentioned, the paper conducts a thorough analysis and provides empirical evidence to support the presented findings. - The rigorous analyses demonstrate a commitment to scientific rigor and methodology. Related Work: - The paper comprehensively reviews the relevant literature, discussing previous works on ICL, language models, optimization, and initialization methods. - The paper differentiates itself from existing works by focusing on the transience of ICL and investigating strategies to mitigate it. - The references and citations are appropriate and up-to-date, incorporating relevant studies in the field. Clarity and Organization: - The paper is well-structured with clear sections and subheadings, making it easy to follow the flow of ideas. - The ideas and concepts are presented logically and coherently, building upon each other to support the main arguments. - The language used in the paper is clear and concise, minimizing excessive technical jargon and enhancing readability. Presentation and Visualization: - The plots are visually appealing, utilizing scalable vector graphics and an attractive gradient color scheme. The figure captions are concise and informative. - The writing is of high quality, with minimal errors or inconsistencies. - The overall presentation is visually appealing and well-designed, enhancing the readability and comprehension of the paper.

Weaknesses

- Lack of experiments on large-scale datasets: The experiments mainly focus on controlled synthetic datasets, and the impact of ICL transience on real-world, large-scale datasets is not explored. Including experiments on diverse, natural language datasets would enhance the practical relevance of the findings. - Limited scope of experiments: The experiments were not performed on large language models, one of the most important applications of transformers. Conducting experiments on larger models would provide a more comprehensive understanding of the phenomenon and its implications. - Narrow definitions of ICL and IWL: The definitions of in-context learning (ICL) and in-weights learning (IWL) used in the paper are relatively specific and make for a clear-cut distinction that in reality - as acknowledged in a footnote - may not be that clear. Exploring additional mechanisms and interactions between context and learning could contribute to a more comprehensive understanding of transformer models. - Limited exploration of alternative explanations: The paper acknowledges possible explanations for the emergence and transience of in-context learning but does not thoroughly explore alternative hypotheses. Further investigation into optimization quirks, initialization schemes, or alternate optimizers could provide deeper insights into the phenomenon. Overall, the above-mentioned points can be seen more as ideas for future work rather than genuine weaknesses of the paper. Minor points: - Lack of statistical significance analysis: Although the experimental results are presented clearly, the paper does not explicitly address the statistical significance of the findings. Including statistical tests or discussing the significance of the observed differences would strengthen the conclusions. - (minor) Maybe explain in-weight learning in the abstract, contrast it to in-context learning - (minor) second to last sentence in the abstract (l. 13), reformulate: how does that question logically follow from the observation? - (minor) l. 18 Reformulate definition to be clearer, order of words - redo the references, missing information (ls. 373, 394, 408), inconsistent formatting (e.g., ls. 307, 315) - Capitalize "In-Context" instead of "In-context" in the title. - Clearer motivation for why researching ICL is interesting and relevant?

Questions

- Were the plots in the paper based on multiple or single runs? If only a single run was used, how was statistical significance ensured? - Have you considered the potential interplay between ICL and other important mechanisms in transformers, such as contextual information used in conjunction with IWL for narrowing down possible outputs via copying? - Figure 5 shows how many runs were averaged, and what do the error shadings indicate? - Could you further explain the statement that the balance between ICL and IWL is a mirror reflection of grokking? Why would ICL, being more generally useful, be transient instead of persisting? - Have you explored alternative regularization techniques or approaches to achieve the coexistence of ICL and IWL? - When the regularization value exceeds 1e-4, ICL transience returns and IWL also does not work. Does this imply a decrease in accuracy with more training? What is the relationship between regularization, ICL, and IWL? - What may be the underlying mechanism by which regularization affects the persistence of ICL? Does it make IWL more difficult or even impossible, leaving only ICL as a viable option? - How do you envision your findings' potential real-world applications or implications on ICL and its transience in transformer networks? - How might this research impact the development and training of AI models in practical scenarios?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

While the potential negative societal impacts are not explicitly discussed in the provided excerpt, the authors have identified areas for further investigation and improvement in their research, which addresses some limitations of their work. Since this work focuses primarily on analytical rather than directly enhancing capabilities, there are no immediate negative effects to be considered.

Reviewer wdqh6/10 · confidence 4/52023-07-06

Summary

The paper investigates the emergence and persistence of in-context learning (ICL) in Transformer models. Training Transformer models on tasks by Chan et al. [10], the authors find that ICL does not persist, despite an initial emergence. Loss reduction suggests a preference for in-weight learning after some training steps. Through specialized experiments, the authors explore model size, training dataset statistics, and L2 regularization to understand and mitigate this phenomenon. The findings emphasize the importance of L2 regularization in Transformers during extended training. This research provides valuable insights into ICL dynamics, contributing to better understanding of this phenomenon and offering insights into potential improved training strategies.

Strengths

1. The paper builds upon the established dataset by Chan et al. [10], ensuring reproducibility of results. The phenomenon of interest is consistently observed across different scenarios, allowing for comprehensive examination. The inclusion of small-scale experiments facilitates further investigation by researchers. The availability of the code, given that it will be released, will help with reproducibility and transparency of the study. 2. The authors adopt a systematic approach to address the problem at hand, systematically eliminating various factors that contribute to the lack of persistence in in-context learning (ICL). Through controlled experiments, they explore and analyse multiple aspects that may impact ICL persistence. 3. The paper has a well-structured and coherent storyline, ensuring clarity of the results. The authors effectively motivate each experiment, providing a clear rationale for their choices. I enjoyed reading the narrative presented by the authors.

Weaknesses

1. The phenomenon is observed for a single small dataset. Although some experiments are made with respect to aspects of the training data, i.e. number of classes and number of exemplars per class, the small relative size of the training data in conjunction with the large number of training steps tested for here, may potentially lead to conclusions that do not generalize past this setting. Some attempts were made to match data distribution with naturalistic data, but any conclusion that generalize to large language models have to be taken with a grain of salt. 2. Although a lot of different effects are explored and some correlations explored, no clear answer is given in the end.

Questions

1. Connections to Grokking are not obvious to me. In Grokking [27] memorization at early phase of training is replaced by more general solutions at later stages of training. In your context, the opposite effect seems to be happening, if you assume that ICL are the more general solutions. 2. What is the effect of different positional encodings to the emergence and persistence of ICL? It would be interesting to study. 3. In Figure 7 what is the "chance" prediction for IWL? Different runs here have different number of classes. 4. You can match the color scheme between Figures 9,10 and Figure 11. 5. Line 242-244 "Furthermore, these results imply that it may be desirable to stop training large transformer models before a compute budget is reached—to preserve ICL—even if training losses are still decreasing.": the statement is a bit too bold given the experimental setup of the paper. 6. Line 258-259 "lottery tickets contained in common initialization methods [27, 37] that incentivize ICL": I find this a very interesting conjecture that could be explored. 7. Another potential effect that could be explored may be the "softness" of the softmax function. Perhaps the network realizes that to decrease the loss further it needs to resolve to IWL. It would be interesting to track the entropy or "hardness" of the attention mechanism. If the softmax is deemed to be the deciding factor, you can think about replacing the softmax with a harder version e.g. [1, 2, 3] and see if the transient effect disappears. [1] Martins, André, et al. "Sparse and continuous attention mechanisms." Advances in Neural Information Processing Systems 33 (2020): 20989-21001. [2] Hahn, Michael. "Theoretical limitations of self-attention in neural sequence models." Transactions of the Association for Computational Linguistics 8 (2020): 156-171. [3] Chiang, David, and Peter Cholak. "Overcoming a theoretical limitation of self-attention." arXiv preprint arXiv:2202.12172 (2022).

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

The paper discusses a limited scenario where ICL is not persistent. Experiments are performed but always in the context of the provided dataset and for smaller models. More experiments are in general required to verify the existence of this phenomenon elsewhere. The study is nonetheless very interesting and useful for the community.

Reviewer iyV56/10 · confidence 3/52023-07-06

Summary

The paper explores In-Context Learning (ICL), a model's ability to adapt its behavior during inference without weight updates, and In-Weights Learning (IWL), where the model relies on information stored in its weights. While early neural networks displayed ICL in controlled scenarios, research changed with the transformer model, particularly with GPT-3, which exhibited ICL without deliberate training efforts. However, it was found that ICL isn't guaranteed when training transformers, with language data properties like burstiness and skewed distribution playing a crucial role. Additionally, ICL and IWL often seem to oppose each other, with ICL emerging more readily when training data has a larger number of tokens or classes and is bursty. Despite the challenges of relying on large models, overtraining is currently a favored approach for training small yet high-performing transformers. The paper challenges the assumption of persistence (that once ICL emerges it remains), showing that ICL can be transient, with the potential to disappear if networks are trained carelessly for longer periods.

Strengths

On the positive side, the paper is certainly quite interesting and relevant to the major open question these days, which is the emergent capability of large language models. The paper raises a number of interesting open questions (see Discussion in section 7). In fact, I would argue that that is the most interesting section of the paper.

Weaknesses

On the negative side, the paper can be viewed as "half-baked" at this point. It relies heavily on some empirical results, derived mostly from small models and visual tasks, and it raises a number of questions that are interesting but.. open. I wish the paper could go a bit more deeply to actually answer at least one of those questions. At this point I cannot say with confidence that the paper is ready to be presented at the premier conference of deep learning.

Questions

My major question is -- is the process of Chan et al. [10], which has been adopted by the authors of this paper, really capable to answer such fundamental questions about ICL and the emergent capabilities of LLMs? The queries that the paper relies on very short -- just 8 exemplar-label pairs and one query image -- i.e., just a sequence of length 17. That also means that the provided "context" is very short. this could be a reason that ICL is not persistent. Another potential issue is the "context" in these experiments is very highly informative regarding the given query. This is why we see that ICL gives a much much larger accuracy than IWL in all plots of the paper. Is this a realistic/pragmatic evaluation regime? I would assume that in practice the context is only partially informative (at least for NLP tasks) and that the network has to rely in anyway much more on trained weights.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The paper is quite open about its limitations, mentioning clearly that it has not utilized very large models and so it is not really certain whether the major claims of this study would be actually still seen in large models such as GPT3 or 4. Additionally the paper does not try to "oversell" what it does -- and it identifies many open questions instead of claiming that it already has answers to those questions.

Reviewer oS5Q5/10 · confidence 2/52023-07-14

Summary

This paper presents empirical evidence illustrating the transient nature of in-context learning (ICL) in transformers, and it shows that the models may shift to in-weights learning (IWL) when overtrained. Furthermore, the transience of ICL is studied across diverse settings such as model size, dataset size, data distributions, and training regularization. Results reveal that adequate L2 regularization can mitigate this transience.

Strengths

1. This paper reveals a characteristic of transformers performing transient ICL. 2. The proposition that regularization can eliminate the transience is noteworthy. 3. The provided experiments are clearly organized and easy to follow.

Weaknesses

1. The paper lacks clarity in its notations, which makes them hard to follow without reference to previous literature. For instance, the term "in-weights learning" needs to be related to supervised learning; in the Zipfian distribution equation $p(X = x) \propto x^{-\alpha}$, the terms $X$ or $x$ should be explained as representing the rank of input classes. 2. While the authors provide numerous empirical evidences, they scarcely discuss potential underlying reasons. Some interpretations or conjectures would enrich the discussion around the observed outcomes. 3. The experiments are constrained to a single dataset and transformer (with different sizes), which is not convincing enough in my opinion. It would be interesting to see if the similar phenomenon is observed with other datasets (e.g., NLP dataset) or other auto-regressive models. The usage of the Omniglot dataset—a stationary supervised dataset—where in-context samples can be regarded as independent entries, makes the existence of IWL unsurprising.

Questions

1. Why are the IWL accuracies consistently low? Would further training iterations increase the IWL accuracy and what is the maximum achievable IWL accuracy? 2. Have the authors considered that the observed ICL performance drop might be due to the fact that evaluation in-context data distribution is different from training, especially as the label space changes? In Fig 5(c), when evaluation and training use the same data distribution, the performance does not decay. 3. The connection between Fig5(a)(b) and Fig5(c)(d) seems unclear given the different training datasets. What's more, in Section 4.1, the authors' claim "Figure 5d shows that larger embedding dimensions enhance the model’s ability to perform IWL, but do not consistently help ICL (Figure 5c)." seems puzzling. Since ICL accuracies are already close to ones, is there room for improvement? Would reducing the embedding dimension identify a threshold beyond which ICL accuracy is $<1$? 4. In the Discussion section, the authors note "However, after training for a long time, we find that even rare classes begin to be learnt in-weights." However Section 4.3 does not provide evaluation over rare classes.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

2 fair

Contribution

2 fair

Limitations

yes

Reviewer iyV52023-08-10

thank you for the thoughtful rebuttal. After also considering the other reviews, I have decided to stay with my original evaluation score.

Reviewer wdqh2023-08-15

Thank you for the additional details. Interesting to see that longer contexts don't seem to affect the emergence of this behaviour. I appreciate the authors for experimenting with different positional encodings. Results seem to indicate that positional encodings may affect the transient nature of ICL in Transformers. More investigation will clarify this in the future. Overall, I find the work a valuable addition towards understudying generalization in Transformers. Some of the results are preliminary and empirical evidence could be expanded. Nonetheless, the overall presentation is excellent. Hope this might inspire future work! I will keep my original score.

Reviewer oS5Q2023-08-20

Thank you for your detailed response and the additional experiments provided. I have a few suggestions for the updated version: 1. Could you investigate if training the transformer for more iterations would lead the model to perform IWL precisely? 2. I'm curious if the observed phenomenon - where ICL emerges first, then fades, giving way to IWL - is due to the dataset's classification difficulty. How would the results differ if the dataset had fewer classes, say 10 or even less? 3. Could you provide evidence that, after more iterations, the rare classes of Zipfian data are indeed learned in weights? A minor suggestion on presentation: Enhance the clarity of your plots, perhaps by rescaling the y-axis (e.g., Fig 9(b)), to better highlight IWL performance. I've decided to increase my score from 4 to 5, and I am inclined towards acceptance.

Authorsrebuttal2023-08-20

Thank you for your remaining comments, and your recommendation for acceptance! 1. We are indeed performing extra long runs to investigate the behavior on IWL evaluation in the long run, partly motivated by some of your questions in the initial review. The results are coming in slowly, but the IWL evaluation has continued to show gradual improvement -- we can definitely include these results when the runs complete. 2. It is a good question re number of classes. As found in Chan et al. (2022), we have also found that a given level of difficulty is necessary for ICL to emerge in the first place. For example with 10 classes, the model never learns ICL, and jumps directly to the IWL solution. We believe that this is because the IWL is too much easier than ICL, in that scenario. These results replicate the interesting results of Chan et al. (2022). 3. We are unfortunately still running these newer Zipfian experiments, so we cannot report on the results. For now, we are removing that statement from the paper. Thank you so much for directing our attention to this question -- we will only re-incorporate that statement once we have definitive evidence from the new experiments. Thank you also for your suggestion regarding the presentation of the plots.

Authorsrebuttal2023-08-20

Thank you so much to the reviewers for all your detailed comments and questions. We believe that this discussion period has substantially improved both the content and clarity of our paper. Thank you also for your unanimous recommendations for acceptance -- we are honored and would be grateful to share our work at NeurIPS this year.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC