Residual Alignment: Uncovering the Mechanisms of Residual Networks

Summary

This work discovers the phenomenon of Residual Alignment (RA) in ResNets, whereby the top left and right singular vectors of residual Jacobians align with each other and in between different residual blocks. Through extensive experimental verification as well as novel theoretical frameworks and derivations, the paper shows that RA naturally emerges in ResNets but does not emerge in non-residual networks. By directly linking RA to ResNets, this work sheds new light on the broad success of these architectures.

Strengths

I believe this paper can have a significant impact. - *Significance*: This work provides novel insights that have wide ramifications in a broad range of topics (deep learning theory, regularization, architecture design, model compression). Indeed, the precise mechanisms underpinning the success of ResNets remains a hot topic of research with high stakes. I believe this work makes a rare breakthrough in that direction. - *Novelty*: To my knowledge, the discovery of RA and its precise mechanisms is novel. There is also novelty in both the experimental protocols and the theoretical frameworks. These protocols and frameworks might prove useful for the community. - *Quality*: The analysis is highly convincing. I feel the experimental and theoretical evidences supporting the analysis are undebatable. - The experimental protocols are innovative and strongly convey the paper's claim, demonstrating both the presence of RA in residual networks and its absence in non-residual networks. - The theoretical analysis is equally innovative and excellent. - *Contextualization*: The context of the work is appropriately provided (aside from minor points detailed below), with connections made to previous works, including to the latest developments on layer-wise Neural Collapse. The paper also details potential ramifications of the discovery of RA. - *Clarity*: The paper is well-structured, concise, clear. The writing is good. The plots and figures are striking, providing the required evidence to support the claims. - *Soundness*: The theoretical proofs are sound (aside from very minor points detailed below). - *Reproducibility*: The authors released their code for reproducibility. The code is neat and clean, with high quality standards.

Weaknesses

I see *no real weaknesses*, only minor points that can be easily addressed. Minor points: - Restriction to the context of classification. RA1 and RA3 are specific to classification, and all the tasks considered fall within such a context. If the findings are restricted to classification, I think this restriction should be clearly stated. - I understand that the class-wise equi-spacing on a line of intermediate representations is a consequence of RA (in particular the scaling of top singular values inversely with depth), combined with the increased magnitude of $h_i$ due to the aggregated summation in ResNets. If this reasoning is correct, perhaps it would be worthwhile detailing it? - The authors missed some previous works that studied ResNets vs non-residual networks at initialization, notably [1], [2] and [3]. - There seem to be typos on lines 160, 239, 243. Also shouldn't "these values" be replaced by "the top-1 singular values" in the caption of Figure 5? - It seems Theorem 3.1 is only proved in the context of binary classification. Perhaps this should be stated. Also, I believe there is a term missing in the Equation following line 11 in the Appendix. That term relates to $h_1 - U_{k,1} U_{k,1}^T h_1$ (as can be seen e.g. in the case where all $S_i$ would be equal to zero). - Perhaps Theorem 2 in the Appendix should state the convention of positive singular values, meaning that inequality on the $\sup$ of the trace would always be guaranteed, with equality obtained if the determinant's sign equals 1. Also shouldn't $L+1$ be replaced by $L$ on line 68? References: [1] The Shattered Gradients Problem: If ResNets are the answer, then what is the question?, D. Balduzzi et al., ICML 2017 [2] Gradients Explode - Deep Networks Are Shallow - ResNet Explained , G. Philipp et al., ICLR Workshop 2018 [3] Characterizing Well-Behaved vs. Pathological Deep Neural Networks, A. Labatie, ICML 2019

Questions

I would be keen to have the authors' opinion on the following points: - What happens outside the context of classification (e.g. in regression, object detection, etc)? - Could you expand on the connection between RA and layer-wise Neural Collapse? It seems layer-wise Neural Collapse was originally found to be present in both ResNets and non-residual networks (notably VGGs). Given your findings concerning the absence of RA in non-residual networks, does that mean layer-wise Neural Collapse can occur without RA? - Could you confirm that RA is still in agreement with the experimental findings in Veit et al., 2016? If so, should Veit et al., 2016 be reinterpreted with a new light? - Could you express your view on the connection between RA and stochastic depth? If all residual blocks have aligned singular vectors, shouldn't the effect of stochastic depth be a plain random multiplicative factor of the logits?

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

4 excellent

Limitations

If the authors' findings are restricted to the context of classification, perhaps this restriction should be more clearly stressed.

Summary

The paper "Residual Alignment: Uncovering the Mechanisms of Residual Networks" explores the underlying mechanisms and success factors of the ResNet architecture, which has gained significant popularity in deep learning. The authors conduct an empirical study by linearizing the residual blocks of ResNet using Residual Jacobians and measuring their singular value decompositions. They introduce the concept of Residual Alignment (RA) characterized by four properties: equispaced intermediate representations, alignment of singular vectors, low rank of Residual Jacobians, and inverse scaling of singular values. The phenomenon of RA is observed in well-generalizing models and is absent when skip connections are removed. The authors also propose a mathematical model that demonstrates the occurrence of RA. *Contribution & significance:* The paper identifies an interesting and seemingly novel phenomenon which they term residual alignment. They show that this observation is on solid theoretical grounds by studying the binary classification problem with cross entropy loss. This sheds new light on an the role of residual connections in neural nets and will contribute to our overall understanding of this importance architectural component. *Major writing issues* The main drawback of this paper is its writing quality which is very low at the moment. While I grasped the main ideas by going through the appendix, several of critical pieces of information, such as definition of residual alignment Jacobians ($J_i$'s), are missing from the main text. This will make it very difficult to understand this paper just by reading the main text (which should be the default assumption for a reader). This is quite unfortunate given that the contributions are technically strong & very interesting. Thus, a significant overhaul of the writing seems to be necessary to make this work publishable. *Clarity about novelty* It would be helpful if authors clarify the new insights and novelty of this work, in contrast to the paper that they numerously cite "A Mathematical Principle of Deep Learning: Learn the Geodesic Curve in the Wasserstein Space". Also adding more related literature will also be helpful to the readers.

Strengths

- Identification of Residual Alignment: The paper identifies and characterizes an interesting and seemingly novel phenomenon of Residual Alignment (RA), and highlights its four key properties. The properties of RA are logically interrelated and consistently observed across various architectures. The authors present evidence to support the existence of RA. - The authors conducted a comprehensive empirical study of the ResNet architecture, linearizing residual blocks using Residual Jacobians and analyzing their singular value decompositions. This approach provides valuable insights into the underlying mechanisms of ResNet models. - Theoretical Proofs and Abstraction: One of the strongest points of this paper seems to be the theoretical proofs for the emergence of RA in the setting of binary classification with cross-entropy loss. The introduction of the Unconstrained Jacobians Model as a mathematical abstraction adds further depth to the analysis and strengthens the paper's theoretical foundations. - The proof the main theorem in the appendix is interesting in its own right. They authors first linearize the loss function and then approximate it as a product of $(I+J_i)$ terms where $J_i$'s are the Jacobians. They go on to argue that the loss of a general configurtion (non-aligned singular vectors) is surprisingly bounded by a term that is equivalent to the aligned Jacobians. This is mainly done by invoking an interesting mathematical theorem. While I did not go into full detail of the proof and there might be details/flaws that I missed, the overall proof strategy makes sense and is very interesting, as it breaks down a very intricate problem to a tractable one. (As a side point, I recommend authors to conduct a few more detailed experimentation on the details of these steps, namely by directly. testing the inequalities they arrive at).

Weaknesses

# Major issues with writing & presentation of results One of the main drawbacks of the current manuscript is its presentation. There are major problems with the flow and writing which hinder my understanding of some details. Another major issues is the lack of enough related works and connection to the existing machine learning & statistics literature. Overall, I would recommend authors to do a major revision of this manuscript to make it publishable and readable. I tried to compile a list of minor issues with the writing to help with this. - Fig 1 caption: “s true label and connected to form a trajectory” every connected line goes from input to output? Shouldn’t it have 34 layers (it looks like fewer dots than 34) - Equation line 34-35: what is $I$? - Equation between 57 & 58, what’s the dimensionality of $ \sigma’(…) $ in this equation? - Non capitalised sentences (examples 68, 70, 74, 92, 93, several more ) - Jacobians are input-dependent, how are the Figures 2 & 3 made? Do the figures correspond to a single sample input? or averaged across multiple inputs? - What does index $i$ in $J_i$ stand for? This seems to be defined in the appendix but in the main text? - line 59 “excluding the contribution from the skip connection.” What does this mean?

Questions

One of my main questions concerns the background of this work. The relevant (recent) literature seems to rather scarce. The only directly related literature currently is "A Mathematical Principle of Deep Learning: Learn the Geodesic Curve in the Wasserstein Space". So my question is two folds: - Do the authors believe there is no other related literature on this phenomenon of "emerging singular vector alignment"? Given the importance of this topic and its relevant to modern models, it seems to me that there should be more "adjacent" studies on the topic. - Given that the aforementioned paper "A mathematical ... " does have quite a bit of overlap with the present work, can the authors make it clear how are their contributions different from this paper? In particular, the residual alignment phenomenon has been extensively discussed in the other paper with largely overlapping settings. It would be helpful for readers to understand what is the additional insights, ideas, proofs, etc, in this present paper.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

1 poor

Contribution

4 excellent

Limitations

yes

Summary

The paper tries to analyze the remarkable performance of ResNet architecture and they find the residual alignment phenomenon. The phenomenon is general and they also proposed a mathematical model call the Unconstrained Jacobian Models to theoretically analyze it.

Strengths

The authors find an interesting phenomenon for residual networks called residual alignments. I think it can guide future exploration for module designing, model compression, and regularization techniques.

Weaknesses

I think the weaknesses of this paper have they only discussed such a phenomenon. It will be great if they can utilize such phenomenon to design some useful techniques.

Questions

1. Is such residual alignment phenomenon a general case in different models with skip connection, like ViT or ResGCN? 2. Are there any techniques or architectures can cause residual alignment and then lead to better performance besides skip connection?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

None

Summary

This paper investigates the ResNet architecture, a popular deep-learning model known for its improved performance through skip connections. The authors aim to uncover the underlying mechanisms behind its success. They conduct an empirical study by linearizing the residual blocks of ResNet using Residual Jacobians and analyzing their singular value decompositions. The measurements and analysis conducted in the study reveal a phenomenon called Residual Alignment (RA), which is characterized by four fundamental properties. First, intermediate representations of a given input are evenly distributed on an embedded line in the space. Second, the top left and right singular vectors of Residual Jacobians align with each other and across different depths. Third, Residual Jacobians are at most rank C for fully-connected ResNets, where C represents the number of classes. Lastly, the top singular values of Residual Jacobians decrease inversely with depth. The study consistently demonstrates the occurrence of RA in ResNet models that generalize well. This phenomenon holds for both fully-connected and convolutional architectures across various depths and widths and different numbers of classes on benchmark datasets. However, RA ceases to occur when skip connections are removed. The authors also propose a novel mathematical model where RA is present. The findings suggest a strong alignment between residual branches in ResNet, imparting a rigid geometric structure to the intermediate representations as they progress linearly through the network until they reach the final layer, where Neural Collapse occurs.

Strengths

Originality: Using the idea of linearization of the Residual Jacobian is commonly known for exploring the behavior of Residual Networks during the training and even pre-training. The authors use the idea of the Unconstrained Jacobian Model, which is interesting but also limited to a binary classification task. Still, it is novel to the best of my knowledge. Quality: The motivation and idea are clearly defined, and experiments support the RA phenomena. Clarity: The paper would benefit from improved organization, such as relocating the related work section from section 5 to section 3 or 2. Additionally, the appendix appears disorganized, with figures located randomly. Other than these, it is written very well and easy to follow. Significance: The study's results demonstrate the presence of the Residual Alignment (RA) phenomenon in the singular value decomposition (SVD) of the linearized Residual Jacobian. The mathematical analysis further confirms the occurrence of RA in binary classification. However, the paper lacks a deeper exploration of RA (2-4) and other potential insights.

Weaknesses

Linearization of the Residual Jacobian and analysis of its SVD decomposition: Despite being a valid idea, is not novel. The authors could also potentially investigate the distribution of singular values with the help of random matrix theory. These singular values depend on the input feature vectors (or hidden representations), but this dependency is not explored. Limited applicability to binary classification: The use of Unconstrained Jacobian Models, while interesting, is deemed limited in the context of binary classification tasks. The paper does not explore its potential beyond this specific task. Organization and clarity: The paper would benefit from improved organization, particularly in the placement of the related work section, which could be better situated in an earlier section. The appendix also lacks clear structure and organization, with figures placed randomly throughout.

Questions

* It would have been great to demonstrate the relationship between RA and Neural Collapse a little bit more through experiments. * Exploring the behavior of the singular values’ distribution can provide more insights into RA.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

The authors do not provide potential limitations of their work.

Rebuttal update

I thank the authors for their detailed response, which I highly appreciated. The additional experiments conducted by the authors are enlightening. Specifically, the fact that stochastic depth reinforces RA by discouraging heterogeneous residual Jacobians $J_i$ is compelling. As pointed out by another reviewer, it might still be advantageous to clarify the dependence of Jacobians on inputs:   - If that dependence is overlooked, does it mean that input dependence is assumed to be weak? - In the Figures that explore the dependence between the rank of Jacobians and the number of classes (i.e. outlining a tie between Jacobians and data distribution), are Jacobians computed for a single input, or are they averaged over the whole data distribution? Overall, I would like to keep my rating and remain highly positive.

Response to authors' rebuttal

I thank the authors for their thorough responses. I think the revision significantly improves the quality of the paper. I think the paper is a nice contribution to the field, and have increased my score to Accept.