The Quantization Model of Neural Scaling

Summary

This paper proposes a possible mechanism that explains both the phenomenon where the cross-entropy loss of large language models (LLMs) decreases as a power law with respect to the training corpus size, and the phenomenon in which certain capabilities of LLMs emerge spontaneously as the loss becomes low enough. The authors empirically demonstrate that in a toy problem called "multitask sparse parity", in which their assumptions explicitly hold, an MLP network indeed obeys both the scaling laws and the emergence phenomena. Finally, the authors propose an empirical method to auto-discover skills in LLMs and apply this method to provide empirical evidence that their assumptions also hold in LLMs.

Strengths

1. The paper proposes a novel and timely explanation for both the scaling laws phenomenon and the emergence phenomenon in large language models (LLMs). 2. The authors support their explanation with a clear demonstration on a toy task of multitask sparse parity. Essentially, they trained a simple MLP network on multiple sparse parity tasks and demonstrate both the scaling laws phenomenon and the emergence phenomenon when the distribution of the different tasks follows a power law. 3. In addition, the authors demonstrate the relevance of their explanation for LLMs by proposing a novel method for auto-discovering skills in LLMs and show that the discovered skills obey power law frequencies. Using and scaling this method might be of independent interest for both the mechanistic interpretability community and for designing better pre-training datasets. 4. The paper is clearly written and accessible to readers with varying backgrounds.

Weaknesses

There is insufficient evidence to conclusively prove that the Quantization Model accurately depicts the training of large language models.

Questions

Could you provide a practical demonstration of the Quantization Hypothesis in a controlled environment that is neither overly simplistic nor complex, while ensuring a clearer definition of natural language quantas? For instance, perhaps you could utilize a pre-trained language model like GPT-4 to create datasets with well defined quantas in a similar manner to the TinyStories dataset [1]. [1] Ronen Eldan, Yuanzhi Li (2023). TinyStories: How Small Can Language Models Be and Still Speak Coherent English?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

3 good

Limitations

The authors were very honest in acknowledging less plausible alternative explanations for their empirical findings.

Summary

This paper proposes a hypothesis that there exists a universal and crucial discrete set of computations (quanta) for reducing loss in various prediction problems, and the model's performance is determined by successfully learning these computations. Through this hypothesis, it demonstrates the relationship between power law neural scaling and the effective learning of more quanta, which cannot be solved solely by the memorization ability of the existing model, particularly in complex problems. Additionally, the paper proposes "Quanta Discovery from Gradients" as a method to auto-discover quanta.

Strengths

1. Based on the assumption that there exists a discrete set of computations, known as "quanta," which determines the performance of the model, the paper provides hints for understanding the emergent behavior observed in the scaling law of the model. 2. The objective of the study is to interpret power law scaling and emergent behavior phenomena by using a multitask sparse parity dataset that cannot be solved solely by the model's memorization ability. 3. Through experiments conducted on the multitask sparse parity dataset, the paper shows that when the frequency of using quanta follows a power law distribution, power law neural scaling can occur as the model learns more quanta.

Weaknesses

1. The paper lacks a clear definition of quanta, which could benefit from further elaboration and clarification. 2. Insufficient explanation is provided regarding the criteria used to determine quanta through the Quanta Discovery from Gradients (QDG) method. 3. Extending the experiments on the Quantization Hypotheses, using the proposed toy dataset, to Language Models (LLM) is hindered by the lack of clarity regarding the relationship between LLM tokens and quanta. 4. Section 5 claims that "Clustering based on inputs or outputs therefore seems unlikely to discover quanta in language modeling," but lacks substantial empirical evidence to support this statement. 5. The explanation concerning the relationship between quanta and gradients, which forms the basis for determining quanta in the QDG method, needs to be further developed and elaborated upon.

Questions

1. In Figure 1, what decoding method was used for the auto-discovered quanta? 2. The experiments on the toy dataset appear to be influenced by the decoding algorithm (such as greedy, beam search, random). Is there any consideration given to this aspect? 3. In Figure 4, what is the relationship between the highlighted red section in the prompt and each graph? 4. In Section 5, it would be beneficial to include details about the dimensional of the gradients used in the Quanta Discovery from Gradients (QDG) method. The absence of explicit notation indicating the shape of each symbol for explaining QDG makes it challenging to ascertain the computational complexity involved in implementing QDG.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

2 fair

Contribution

3 good

Limitations

There is a need to demonstrate the effectiveness of the proposed Quanta Discovery from Gradients (QDG) method in identifying quanta in various Language Models (LLM) and datasets.

Summary

This paper proposes a new way of understanding scaling laws. Namely, it shows that capabilities can be thought of as being broken into discrete units (quanta) which themselves follow a power law and have an inherent ordering--called the Q Sequence. This combined with the fact that 1) an increasing subset of quanta are learned over various scales and 2) model and data sizes can be related to the number of quanta gives rise to the scaling laws shown in previous works. The paper also proposes a method to cluster examples by unit-normalized gradient, and empirically confirms within margin of error that theory agrees with observed data. The quantization model explains a sudden emergence of new capabilities with scale: certain quanta must be learned before the model can carry out certain tasks.

Strengths

Simple, effective model to understand scaling laws

Weaknesses

While the paper shows that the model scaling trend agrees with theory on real data, it does not empirically validate whether the same alpha of 0.083 matches the data scaling theory. Also, it would be interesting to show this holds on non LLM tasks like image classification, since the (titular) claim is "neural scaling".

Questions

Everything is clear.

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

4 excellent

Limitations

Not that I am aware of

Summary

The authors investigate neural scaling laws and propose an explanation for the power law scaling that is observed in addition to the emergence of new behaviours.

Strengths

The ideas are interesting and I found some of the experiments such as per token losses on the language models to be quite interesting.

Weaknesses

I found the presentation of Section 5 to be a bit confusing. I also found the connection to emergent behaviours to be a bit speculative. Specifically I think the idea of looking at per token losses/etc to be quite interesting, but I wouldn't necessarily say that I am convinced that this is emergence in the same way I'd think of emergence on some BigBench tasks, for example.

Questions

Could you elaborate on why you feel that this demonstrates emergent capabilities in language models? You mention the spectral clustering hyperparameters: how sensitive are other parts of this work to broad choices of hyperparameters? For example how are you setting batch size / LR for the LM experiments? In Figure 5 right panel there is some interesting shape in the yellow curves-- I'm curious what you think of those?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Yes, the limitations are mostly well addressed.

Summary

The paper proposes that the capabilities of a neural network are composed of discrete “quanta” that each help reduce loss on a subset of training examples, and that are learned in approximately decreasing order of frequency. If the quanta are present in a Zipfian (power-law) frequency distribution in the training data, and each quantum is approximately equally valuable in the examples where it’s present, such a theory would provide a mechanistic explanation for power law scaling of loss with respect to model and dataset size. First the authors validate this hypothesis on a toy dataset constructed to have this quantized structure (by being composed of several independent tasks each with a different prefix key). Then they find similar patterns in real language models, and use an algorithm they call “quanta discovery with gradients” to identify potential quanta in these models and their data. Their results are supportive of the quantization hypothesis, but with a high degree of uncertainty.

Strengths

The hypothesis is a beautiful and important claim if true, and the paper provides a well presented and solid chunk of evidence that it is. The toy dataset experiment is simple and straightforward and demonstrates that the quantization hypothesis works under ideal conditions, and the LM experiment shows that similar phenomena are also present in real models, while proposing a reasonable initial approach to quanta discovery. The results on monogenic vs. polygenic samples also point pretty clearly towards the validity of quantization. Given QDG or any other model of what specifically the quanta might be, the hypothesis also enables testable predictions about scaling laws.

Weaknesses

The multitask parity setting seems too obviously likely to lead to quanta, so it doesn’t seem to prove very much (although it’s useful to set up the framing, almost like a thought experiment). The QDG methodology is also a bit disappointing in a few ways. By clustering the gradients, it assumes that quanta (conceptually defined without reference to model structure) will be localized in the model—which is probably true, but might otherwise have been a testable hypothesis. It’s also too slow for the authors to have applied it to models larger than the smallest Pythia LM, greatly limiting its applicability.

Questions

How much do you believe the hypothesis overall? Might there also be a significant role for “interaction terms” between quanta or are they close to being fully independent? Do you have early ideas about alternatives to QDG, especially alternatives that might be more computationally tractable?

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

4 excellent

Limitations

The authors acknowledge the limitations of their methods and experiments.

Thank you

Thank you for the thorough rebuttal and additional experiment you performed. After reading the other reviews, I have decided to keep my score unchanged.