Summary
The paper studies the generalization abilities of a next token prediction pre-training with Transformers architecture to solve new unseen tasks. It performs a series of controlled experiments on simple multivariate regression tasks. The authors show that, unlike suggested in previous works, the solution learned by the transformer architecture deviates from that of the Bayesian solution with finite-support prior over tasks (dMMSE) and converges to the Ridge solution in the finite-set regime past a certain threshold. They empirically characterise when this transition occurs and study how different factors influence it.
Strengths
1. The paper is well-written and introduces all the necessary background needed for understanding.
2. The paper studies the generalization of transformers pre-trained with next token prediction to solving novel tasks in-context in a few-shot manner which is an important direction to understand how models such as GPT generalize to solve language tasks seemingly different from what they were trained on.
3. The empirical study is well controlled and supports the conclusions made.
Weaknesses
1. While described by a new term, “in-context learning” (ICL), the setting studied in the paper is practically identical to that of few-shot and meta-learning. Therefore, it is important to position w.r.t. this line of work, including the studies on the role diversification of pre-training tasks, e.g., [1-3]. See also [4] for theoretical foundations on learning to learn.
2. The problem of underfitting is not studied sufficiently well. It is mentioned in the paper that, in the main experiment (Fig. 1), the PT model underfits as the error on the training tasks is higher than that of the optimal solution. This would be an important aspect to clarify as one would expect that a model with sufficient capacity should converge to the optimal solution on the training dataset, i.e., dMMSE.
[1] M. Yin, G. Tucker, M. Zhou, S. Levine, and C. Finn, “Meta-Learning without Memorization.” arXiv, Apr. 27, 2020. doi: 10.48550/arXiv.1912.03820.
[2] R. Kumar, T. Deleu, and Y. Bengio, “The Effect of Diversity in Meta-Learning.” arXiv, Nov. 24, 2022. doi: 10.48550/arXiv.2201.11775.
[3] K. Hsu, S. Levine, and C. Finn, “Unsupervised Learning via Meta-Learning.” arXiv, Mar. 21, 2019. doi: 10.48550/arXiv.1810.02334.
[4] J. Baxter, “Theoretical Models of Learning to Learn,” in Learning to Learn, S. Thrun and L. Pratt, Eds., Boston, MA: Springer US, 1998, pp. 71–94. doi: 10.1007/978-1-4615-5529-2_4.
Questions
1. (See point in weaknesses for more context). Is the underfitting seen in Fig.1 top left due to the evaluation on the new data points (for the same tasks) or insufficient model capacity? If latter, did authors try to increase capacity with parameters other than the dimensionality of the embedding in Sec 3.3, e.g., the depth/number of attention heads? It would be interesting to see whether there is an implicit bias in the transformer architecture (trained with SGD) regardless of its capacity.
- Note that this observation also contradictions to the statement made in L124: "A PT that minimizes the pretraining loss LTPretrain will behave like this estimator."
2. What are “fundamentally” new tasks? This term is mentioned multiple times, but it is not defined what makes some tasks “fundamentally” new. Moreover, the new test tasks are sampled from the same distribution as the training ones ($w_i \sim N(0, I)$). See the next question for a broader discussion.
3. What makes the observed behavior (the transition from dMMSE -> Ridge) emergent and different from a superior iid generalization of one algorithm (next token prediction with Transformers) over the other (Bayesian dMMSE)?
- In standard supervised learning, iid generalization occurs from training to test examples sampled from the same distribution. In this case, one learning algorithm, e.g., ResNet trained with SGD, can exhibit better iid generalization than another, e.g., AlexNet, i.e., converges to the true solution with fewer training examples.
- Similarly, the paper shows that one (meta-)learning algorithm (next token prediction with Transformers) is superior to the Bayesian dMMSE algorithm.
- More specifically, the setting of ICL can be described in terms of standard supervised learning (perhaps, with varying dimensionality of input). Indeed, the input can be defined as $x = S_k$ and the output as $y_k$, then training examples are $(S_k, y_k)$.
- Are these analogies (fairly) the same, or is there a fundamental difference?
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Limitations
1. Similar to the raised question about the similarity to a standard iid generalization setting, the experimental setup of this paper does not consider the distribution shift occurring between train and test tasks, an important aspect of generalization (most probably) occurring between LM pre-training and natural language tasks.
- Sec 5 of the paper mentions that the diversity of the pre-training dataset of tasks is important. However, this is only supported when more tasks are sampled from the same distribution as test tasks ($w \sim N(0, I)$ in both train and test). Whereas in language, there is (most probably) a distribution shift between them.
2. The paper limits the study to transformer architectures. It would be interesting to see how other architectures used in few-shot and meta-learning behaves in the same settings, e.g., [1]
[1] M. Garnelo et al., “Conditional Neural Processes.” arXiv, Jul. 04, 2018. doi: 10.48550/arXiv.1807.01613.