We investigate the spectral properties of linear-width feed-forward neural networks, where the sample size is asymptotically proportional to network width. Empirically, we show that the spectra of weight in this high dimensional regime are invariant when trained by gradient descent for small constant learning rates; we provide a theoretical justification for this observation and prove the invariance of the bulk spectra for both conjugate and neural tangent kernels. We demonstrate similar characteristics when training with stochastic gradient descent with small learning rates. When the learning rate is large, we exhibit the emergence of an outlier whose corresponding eigenvector is aligned with the training data structure. We also show that after adaptive gradient training, where a lower test error and feature learning emerge, both weight and kernel matrices exhibit heavy tail behavior. Simple examples are provided to explain when heavy tails can have better generalizations. We exhibit different spectral properties such as invariant bulk, spike, and heavy-tailed distribution from a two-layer neural network using different training strategies, and then correlate them to the feature learning. Analogous phenomena also appear when we train conventional neural networks with real-world data. We conclude that monitoring the evolution of the spectra during training is an essential step toward understanding the training dynamics and feature learning.
Paper
Similar papers
Peer review
Summary
This paper studies gradient descent training in single-hidden-layer neural networks in the linear-width regime (i.e., that in which the input dimension, hidden layer width, and number of training datapoints together tend proportionally to infinity). Its primary result is that the bulk spectra of the conjugate and tangent kernels do not change over training in this regime, but at large learning rates an outlier eigenvalue can emerge.
Strengths
The topic of how kernels evolve outside of the lazy regime is timely, and the RMT-inspired setup and results presented should be of interest to the community. The theoretical results appear correct, though I've not checked the proofs line-by-line.
Weaknesses
The manuscript presents a series of interesting observations, but to my taste it sacrifices depth for the sake of breadth. The results on heavy-tailed spectra are thought-provoking, but I think focusing on the emergence of spikes in the spectra would make for a stronger manuscript. If the authors could prove the existence of a BBP-like transition, they would have a strong paper. The current comment on Lines 247-248 comparing to [6] is not very satisfying. At the very least, a more detailed empirical investigation of this phenomenon would result in a more convincing story. In this vein, the experiments are not very systematic, and as a result cannot clearly disentangle which changes in the setup are required to produce a given phenomenon. For instance, Table 1 contains various settings of batch size and learning rate, but nowhere are these parameters swept jointly over some reasonable range. I have in mind something like the parameters sweeps for learning single-index models performed in recent work by Atanasov et al., "The Onset of Variance-Limited Behavior for Networks in the Lazy and Rich Regimes" (ICLR 2023; note also that this relevant recent work is not cited). The goal here would be to establish under precisely what conditions a spike emerges. As it stands, the authors can make only vague statements. The transition to using Adam for the heavy-tailed weight experiments is even more drastic: is it possible to observe this phenomenon using a non-adaptive optimizer? I think dissecting this phenomenon is more properly the subject of a separate paper, as it is currently not well-integrated with the other results. The discussion in Lines 286-291 is also rather heuristic, and I don't think the authors provided sufficient evidence for their claim that heavy-tailed spectra naturally emerge in feature-learning networks trained on complex tasks. Moreover, how does "[t]his example explain[] why we can use the heavy tails to discriminate well-trained and poorly-trained large models?" This requires elaboration, and more empirical evidence.
Questions
- In the Introduction, it would be nice to mention recent work on Gaussian equivalence for deep random feature models in the linear-width regime by Schröder et al., "Deterministic equivalent and error universality of deep random features learning" (ICML 2023) and Bosch et al., "Precise Asymptotic Analysis of Deep Random Feature Models" (2023). - The linear-width regime for deep Bayesian neural networks has recently been the subject of study in the community of researchers using tools from statistical physics to study deep learning, see e.g., Li and Somplinsky, "Statistical mechanics of deep linear neural networks: The backpropagating kernel renormalization" (PRX 2021), Zavatone-Veth et al., "Contrasting random and learned features in deep Bayesian linear regression (PRE 2022), and Cui et al., "Optimal learning of deep random networks of extensive-width" (ICML 2023). I think it would be useful to at least comment upon the Gaussian equivalence results of Cui et al. as a point of reference for how different inference procedures behave in the extensive-width regime. - In Figure 2 (b-d) and elsewhere, it would be better to label the abscissa with $\eta$ or "learning rate" rather than "lr," for the sake of consistency with the text. - In Figure 1 and elsewhere, the histograms should be plotted using a log-scaled ordinate. As it stands, the outlier in 1b is nearly invisible. - Why use the single-index + quadratic model in eq. 8 rather than a (simpler) single-index model? - The BERT experiments in Figure 5 strike me as mostly decorative, as the setting is not commensurate with the earlier portions of the paper. What insights would the authors argue can be transferred from their MLP experiments to this setting?
Rating
4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
2 fair
Presentation
2 fair
Contribution
2 fair
Limitations
The paper does not provide adequate discussion either of the relevance of the linear-width setting for practice or of the limitations of their experiments.
Summary
This paper examines the spectrum of the conjugate kernel and neural tangent kernel of real networks before and after training. Based on analyzing the spectrum in some experiments, several observations about how the spectum evolves during training are made. In particular, there seems to be different phases depending on the optimizer used and learning rate which effect how the spectrum changes from initialization. Understanding this spectrum-evolution phenomenon may lead to a better understanding of how neural networks train, and shed light on important questions like understanding differences between ADAM vs GD vs SGD.
Strengths
* The review of existing literature is really well done (there is even a more detailed review in the appendices). This puts the current work in context quite nicely and makes this paper a very accessible place to start learning about these spectral questions. * The experimental plots are nicely done and clearly showcase the behaviour being discussed (the Q-Q plots also help a lot with the understanding). There are also numerous other experiments in the appendix. This again makes this paper a great resource for these kinds of experiments. * I am not an expert on this area, but to me the way the different phases of spectral evolution are laid out (i.e. what they call "Invariant Bluk" vs "Bulk+Spike" vs "Heavy tail") is new to me. (I have heard of this outlier thing happening before for the Hessian...I am not sure how related that is). To me this paper looks like the first step towards building a phase diagram of how choices about optimizer/trianing effect the spectrum which would be a really useful thing to have. * I found the conclusion about learning rate (e.g. Figure 2) the most impressive conclusion here, and to me this may be the most actionable of all the various phenomenon observed here. This is a nice conclusion to draw and could pave the way for more results in this direction.
Weaknesses
* One might complain that the theory developed here is a bit weak, in that the theoretical results are all inequalities which are (probably) not sharp. However, since the paper is mainly empirical I actually think these theoretical results are reasonable here. * I am not an expert on this area, so I was not able to tell how much of the results here were expected/known already. Hopefully at least one other reviewer is able to give that context. * It is not clear how robust the experimental results are: in the given plots the phenomenon seem to be happening as described, but maybe there could be some extra discussion about "edge cases" between the various phases and discussion about when things break down (e.g. when N is too small, or there are not enough examples etc)
Questions
* Is there any possibility to describe how the outlier eigenvalues are evolving in time during training? Particularly when there is only one outlier eigenvalue, this would be an interesing bit of theory to try and understand. * Is there any relationship between the spectrum of the CK/NTK and the spectrum of the Hessian?
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
3 good
Contribution
3 good
Limitations
* One limitation that always comes up in these limiting type things is the question of how large real neural networks have to be for the theory to actually work. Some discussion or experiments specifically addressing this (e.g. showing the error in the predictions as a function of network size) could help explicitly address this.
Summary
The paper carries out an empirical study of the spectral properties of finite-width neural network kernels after training, and compares them to their infinite-width counterparts to assess if there is an improvement. They study these changes with respect to various hyperparameters such as learning rate and optimizer. The results suggest that for large learning rates, where the linear dynamics is not a valid approximation, they show that feature learning emerges by inspecting the distribution of the final spectra. They identify two different spectral property that distinguish feature learning from lazy regime: emergence of low-rank spikes in the spectrum and heavy-tailness. They explore when these properties emerge and provide theoretical arguments for these observations.
Strengths
The paper is well-written and easy to follow. The experiments are complimented well with theoretical arguments, and the discussion is sound. The Supplementary Material also includes nicely organized discussion of previous works and additional experiments and discussion. In summary, they introduce the well-known fact that the small-learning rates make a NN with NTK parameterization stuck around its initialization and derive theoretical results to show that the bulk of the spectrum remain invariant under lazy training. In my opinion, the main contribution is that they find two different ways feature learning can happen: emergence of low-rank outliers in the spectrum with large learning rates with GD optimizers and heavy-tailed spectrum with adaptive optimizers. Furthermore, for GD/SGD, they make a theoretical connection for how low-rank outliers emerge with a phase transition in RMT (ref. [9] in main paper.) Finally, the relation of heavy-tailed spectrum to generalization has been discussed.
Weaknesses
1. While the paper brings valuable insights, it is highly restricted by the 2-layer neural networks and synthetic dataset with a particular target. It would be useful to perform experiments on simple real datasets such as MNIST to show that spikes related to number of classes also emerge in the spectrum. For example, in neural collapse phenomenon, one can see that weight matrices of last layer acquire low rank structure that can be tracked to the number of classes. 2. Such experiments can also shed light on how full-batch vs. stochastic GD differ in terms of the after-training spectrum. Currently, I do not think there are any insights related to how SGD helps feature learning since GD experiment does not have a large LR analogue. 3. Is there an intuitive explanation how adaptive optimizers cause heavy-tailed spectrum in CK? More discussion around this phenomenon could be very helpful.
Questions
1. It is hard to parse Figure 1, first panel shows weight spectrum for GD with LR 5, second panel shows CK spectrum for SGD with LR 22 and last one shows CK spectrum for ADAM. Something like CK spectrum for Case 2, 3, 4 would be more helpful. 2. In Figure 4, is there a reason why GD does not show the same properties as SGD? With large enough LR, could GD also show the same heavy-tailness? Also, it is not clear how adaptive optimizer qualitatively differs from SGD.
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
3 good
Contribution
3 good
Limitations
A limitations section is missing.
Summary
This paper analyzes the spectral properties of feedforward neural networks under a student-teacher setting. A linear-width regime is considered, where the sample size and network width grow comparably with the input feature dimension. The paper shows that the spectra of weight and kernel matrices are invariant under training, and that outliers occur in large step size regimes. Heavy-tailed spectra are also discussed with their relation to generalization. Evidence from both synthetic and real-world datasets is shown.
Strengths
- Proposed LWR as a novel perspective for analyzing feedforward networks, and presented an interesting analysis of NN spectral invariance, potentially providing insights into NN feature learning. - Evidence from both synthetic and real-world datasets enhance the argument. - The presentation of the paper is clear.
Weaknesses
- The proposed LWR feels under-explained. Please see "Questions". Otherwise I think this is a solid papar.
Questions
- In the proposed LWR, how should one understand that the input feature dimension $d$ goes to infinity? This feels somewhat counterintuitive, compared with e.g. the infinite-width regime. - To what extent does the observed spectral invariance depend on the LWR? I would assume this does not occur under the infinite-width regime.
Rating
5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
4 excellent
Contribution
3 good
Limitations
n/a, theoretical work
Summary
Considerable attention in deep learning theory has been given to the analysis of neural networks in the kernel regime, which neural networks approach as they approach infinite width. While this analysis has led to insights, there is an important gap between the theorized generalization performance of neural networks in this infinite width regime and the practical performance of finite-width neural networks, which typically perform better. This paper studies simple, two-layer neural networks in the more realistic *linear-width regime* (LWR), in which sample size, input feature dimension, and the width of the layer approach infinity at comparable rates. In several analyses, the authors target the following question: *How do the spectra of the NN's weight and kernel matrices evolve during the training process?* The authors compare to the kernel regime as a benchmark and analyze changes in spectra that occur through training with different gradient descent algorithms . For their first simple experiment (Section 3), they find that the spectra of the weight, conjugate kernel (CK) and neural tangent kernel (NTK) matrices remain invariant during training for gradient descent (GD) and stochastic gradient descent (SGD) with a low learning rate. This is consistent with the Lazy Training (LT), and indicates that the network is not outperforming a kernel machine. However, for SGD with a large learning rate, a spike in the spectral distribution emerges after training, which indicates improvement over lazy training. They note that this spike is consistent with previously published results that demonstrate that learned features may be indicated by an outlier in the spectra with a large singular value. They classify the spectral distributions into three categories: invariant bulk, spikes outside the bulk, and heavy-tailed distributions. They prove that the invariant bulk phenomenon is expected under certain assumptions. They then empirically analyze the spike phenomenon, and demonstrate that as the learning rate increases, there is a threshold at which the spike emerges, which corresponds with greater alignment and suggests feature learning. They then provide an explanation for the relationship between heavy tailed spectra and better generalization performance. Lastly, they perform some analyses on CNNs trained on more a more natural dataset, CIFAR-2, as well as BERT with fine-tuning on Sentiment140.
Strengths
The authors tackle an important question, which is to understand how the structure of neural networks change through the training process. To get a handle on it, they analyze the spectra of the weight and kernel matrices of simple neural networks on controlled datasets. They find some interesting phenomena that relate the spectral properties to learning rate and generalization performance.
Weaknesses
The paper would benefit from more effort put towards making clear the meaning and significance of the experiments and results. Why look at the spectra? What is the significance of the three classes of spectral distributions that are observed? (The answer to the latter question is in the paper, but it is somewhat buried. This should be put front and center and made very clear). The figures and captions in the paper could be clearer. Oftentimes, variables are used as labels. The reader forgets which variables correspond to what. It would be better to use English names, or more descriptive properties. The specific primary findings should be stated more clearly. The statements provided in the introduction are general, and don't provide much insight into the conclusions. It takes work for the reader to tease out these conclusions.
Questions
- What can we understand about neural networks, generally speaking, by looking at the spectra of their weight matrices and kernels? Why use this approach, rather than analyzing other properties of the weights and network structure? - How do the results found here advance our theoretical understanding of how deep networks learn? - How can these results be used in practice?
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
2 fair
Contribution
3 good
Limitations
The authors note the limitation of the FWR. They do not state limitations of the spectral approach to analyzing networks.
I thank the authors for their detailed response to my concerns and those of the other reviewers. I maintain that the clarity of the paper could be improved by providing more detailed experiments investigating a smaller set of phenomena. This is related to the comment made by Reviewer C57a regarding the need to `zoom in' to the transition region; if the authors can provide a detailed investigation of this phenomenon, it would greatly enhance the paper. Respectfully, I do not think the BERT plots convincingly demonstrate the claim the authors are trying to make. I do appreciate the authors' efforts---I think the changes proposed will improve the paper---so I will raise my score.
Thanks for the reply
Thanks for the reviewer's response and valuable suggestions. Here are some additional responses. 1. In this paper, we showed three different spectral evolutions (invariance, emergence of spikes, and heavy tails). We proved the spectral invariance in certain regimes and presented additional experiments on the other two phenomena. From a spectral analysis perspective, we believe it becomes more complete and all these phenomena are indispensable in our paper. Because of paper limitations, we did not provide enough analysis for all three phenomena. But definitely, in our revised appendix, we will provide more experiments on the transition phenomenon for the spikes in the spectra. Following the figures in the attachment of the general rebuttal, we will show a detailed investigation of this transition phenomenon, especially how the learning rate affects the spike's alignment and test error in the transition region. 2. For the BERT model, we do not want to draw any conclusion but aim to draw the attention of RMT and statistics communities, emphasizing the importance of spectral analysis for even large language models. And we possibly can use spectral analysis to understand how the model learns features. Also, like Figure 4, we want to show that our LWR can be considered in more practical models. It is possible to apply RMT to analyze these large models for future work.
I thank the authors for the thorough rebuttal. My primary concerns with the paper regard clarity. Other reviewers have noted similar challenges with parsing the plots and remarked that the paper "sacrifices depth for the sake of breadth." I believe the results are important and that the work is sound. I agree with reviewer zoyL that, for my taste, I would prefer to see a more focused set of experiments on a smaller set of phenomena. Nonetheless, given the scope that the authors have chosen to present in this paper, I believe the changes proposed in the rebuttal will improve the paper, and I have increased my score accordingly.
Thanks for your feedback
Thanks for your thoughtful response and valuable suggestions. We will balance the depth and breadth in our final version of the paper. In our revision, we will provide more experiments on the transition phenomenon for the spikes in the spectra of kernel and weight matrices. Following the figures in the attachment of the general rebuttal, we will show a detailed investigation of the phase transition phenomenon for spikes, especially how the learning rate affects the spike's alignment and test error in the transition region. This will provide a more focused case study of experiments on our second phenomenon.
I thank the authors for their detailed rebuttal and clarifications. I will raise my score accordingly.
Decision
Accept (poster)