Summary
The paper shows that the apparent suddenness with which capabilities of language models seem to emerge with scale may be an artifact of the metrics by which they are measured. Theoretically, emergence should be more gradual if the metric scales linearly with log loss on some test set. Three sources of empirical evidence are used to support this claim. First, the authors study the scaling of the GPT-3 family on arithmetic tasks and show directly that emergence is smoother when a linear metric is used. Second, they conduct a meta-analysis of BIG-Bench, and show that gradual emergence only occurs with non-linear metrics, and that the effect is reduced when switching to a linear metric. Third, they artificially induce sudden emergence by using non-linear metrics for autoencoders on CIFAR100 and for transformers on Omniglot.
Weaknesses
I think there are notable weaknesses in the empirical evidence given to support the basic point of the paper, and insufficient acknowledgement of the limitations of the evidence and other possibilities that could still give rise to sudden emergence.
On the empirical evidence:
- The main direct evidence that previous claims of sudden emergence are an artifact of the metric comes from studying those previous claims directly, which is done in Section 3. However, the evidence here is somewhat inconclusive: for example for 2-digit multiplication with target str len 4 (which is presumably most 2-digit multiplications?) the jumps from 350M->1.3B->6.7B are gradual while there is a much bigger jump from 6.7B->175B. Moreover, this bigger jump is not predicted by the theoretical model. So it seems like there is still some relatively sudden emergence (even though it is reduced by using the right metric).
- The BIG-Bench analysis is helpful, but again on logical_args it seems like there may still be an unpredictable jump. And on the other tasks, the emergence is not that sudden even with the original metric.
- The induced emergence experiments are also helpful, but they show that sudden emergence can be induced by using a non-linear metric, not that it is always induced by a using a non-linear metric. So this evidence is more indirect.
- Overall I think the paper would have come to more correct (and probably more nuanced) conclusions if it had studied all of the tasks from Figure 1, where sudden emergence had previously been claimed, in detail, instead of studying just one of these tasks combined with other, weaker sources of evidence.
- Nevertheless, for an initial exploration of the topic, I think combining multiple source of evidence, even if somewhat weak, is helpful, and it is reasonable to defer a more careful analysis to future work.
On other theoretical possibilities:
- There is no guarantee that because validation loss scales smoothly, cross-entropy loss on a specific sub-distribution must also scale smoothly. This depends on how the model allocates capacity to that sub-distribution. Therefore there is still an important theoretical possibility of sudden emergence, even with a linear metric. Indeed, this may explain why the data did not match the theoretical model in Section 3.
- In some cases, it may not be possible to construct a linear metric. For example, for program synthesis, whether the program passes the unit tests cannot be easily converted to something like edit distance or log loss.
- It is argued that log accuracy can be used in this case. But this is not necessarily feasible. For example, if current models pass unit tests with probability 1 in a trillion, it is not feasible to measure this accurately enough to perform any useful extrapolation. Therefore there may be sudden emergence for all practical purposes.
Overall, I thought that the evidence was strong enough to support a conclusion that sudden emergence is partially explained by the choice of metric, and that researchers should prefer metrics that scale linearly with log loss when available. I also think that this conclusion is itself enough for the paper to be an important contribution. But I think the paper should take more care not to overstate its claims and acknowledge the limitations of the evidence and, space permitting, other theoretical possibilities. In particular, claims like "metric choice is likely wholly responsible for emergent abilities" (p. 9 line 204) are not justified and should be removed or reworded.
I am recommending acceptance of this paper because of the importance of the contribution, but I think that some effort should be made to revise the paper prior to acceptance. However, I am confident in the ability of the authors to do this.
Some presentation points:
- I think it would be clearer to talk about "sudden emergence" and "gradual emergence" rather than "emergence" and "no emergence", since "emergence" can also be interpreted as "appearing despite not being explicitly trained for". However, I understand that the current terminology is taken from the existing literature and the paper cannot be blamed for this.
- I suggest changing the title of the paper, because the current title can be misinterpreted as asking whether the _abilities_ are mirages, rather than whether the emergence (or sudden emergence as I prefer to put it) is a mirage. Any researcher who reads the abstract should not be misled, but unfortunately provocative titles that can easily be misinterpreted can be picked up on by the media and end up misleading the public. However, I acknowledge that choosing titles is hard and no title is perfect.
More minor points:
- p. 4 line 78 - I am not familiar with this Delta notation (or I have a vague recollection of it). It is clear from the context so probably fine, but consider alternative notation/clarifying the definition.
- p. 5 line 115 - Why choose this task from among all the tasks in Figure 1, and why 2-shot / 4-digit? (Would be even nicer to include results on other tasks/other numbers of digits, but absent that, it would be good to clarify how you chose, to make it clear it was not cherry-picked.)
- Figure 3: Target str len 1: how do you get a 1-digit answer from multiplying two 2-digit numbers? I assume you are counting things like "01 * 01"? Maybe clarify.
- Figure 3: I think it would be clearer to include titles at the top of each column saying "Mathematical Model", "2-Integer 2-Digit Multiplication Task", "2-Integer 4-Digit Addition Task", instead of putting this in the caption. That way people can easily see at a glance what each plot is showing, without having to dig into the caption.
- Figure 4: Same comment.
- Figure 5: It would be nicer to group the metrics based on their mathematical form. E.g. it is unclear what "gender_bias_score" is. This also might help spot if there are more examples with emergence that didn't have high emergence score.
- Figure 5B is a bit redundant. Maybe just point out in figure 5A which of the 5 did not count as emergence. It's also a bit unclear why this didn't count despite the emergence score.
- p. 9 line 209 - I don't think this work really contradicts broken neural scaling laws. Even if sudden emergence goes away when you change the metric, one might still be interested in forecasting the original metric.
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.