Recent work has revealed many intriguing empirical phenomena in neural network training, despite the poorly understood and highly complex loss landscapes and training dynamics. One of these phenomena, Linear Mode Connectivity (LMC), has gained considerable attention due to the intriguing observation that different solutions can be connected by a linear path in the parameter space while maintaining near-constant training and test losses. In this work, we introduce a stronger notion of linear connectivity, Layerwise Linear Feature Connectivity (LLFC), which says that the feature maps of every layer in different trained networks are also linearly connected. We provide comprehensive empirical evidence for LLFC across a wide range of settings, demonstrating that whenever two trained networks satisfy LMC (via either spawning or permutation methods), they also satisfy LLFC in nearly all the layers. Furthermore, we delve deeper into the underlying factors contributing to LLFC, which reveal new insights into the spawning and permutation approaches. The study of LLFC transcends and advances our understanding of LMC by adopting a feature-learning perspective.
Paper
Similar papers
Peer review
Summary
There is a continuing effort to understand the complex training dynamics and loss landscape of neural networks, and one of the most interesting discoveries is Linear Mode Connectivity (LMC). LMC is the phenomenon that when two different solutions are linearly interpolated in parameter space, the training loss and test loss remain low enough to be similar to the solution. In this paper, the authors propose Layerwise Linear Feature Connectivity (LLFC), a stronger linear connectivity than LMC. LLFC means that the feature maps of all layers of two differently trained solutions are also linearly connected. Through various experiments, they show that LLFC is satisfied when spawning and permutation methods, which are common methods for presenting LMC characteristics, are used. In addition, they advance the understanding of LMC with a theoretical explanation of the underlying factors that make LLFC appear naturally.
Strengths
- The paper introduced LLFC, an interesting phenomenon that extends LMC, linear connectivity in parameter space, to linear connectivity in feature space. - The empirical results are comprehensive and contain the necessary contents for the development of the discussion. - The theoretical analysis provides a convincing explanation for the emergence of LLFC.
Weaknesses
- The authors only ran their experiments on MNIST and CIFAR10, which are relatively easy datasets. It will be important to verify its validity on larger datasets such as ImageNet. Git Re-Basin (Ainsworth at el., 2023) also showed a relatively large loss barrier on ImageNet, so validation on a wider range of datasets is required. ----- (Ainsworth at el., 2023) [Git Re-Basin: Merging Models modulo Permutation Symmetries](https://openreview.net/forum?id=CQsmMYmlP5T)
Questions
- I'm not sure how low the value of the sparsity measure is in Figure 4. Is a value of ~0.3 considered sparse? - In the experiments, the values measured with random variables, etc. are used as baseline, but it would be more accurate to compare them to the values for a model that is not linearly connected. The baseline values currently presented seem to exaggerate the results because they are comparing too different components. - Just a question, do you think the ensemble performance would be improved when ensembling linearly interpolated features with the modes?
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
4 excellent
Contribution
3 good
Limitations
I see no potential negative societal impact from this paper.
Summary
The work identifies layerwise linear feature connectivity (LLFC) and proves LLFC is sufficient for linear mode connectivity (LMC) between neural networks. Experimental evidence using two methods for finding LMC networks (spawning and permutation) finds that LMC and LLFC co-occur in a variety of models and datasets. Two conditions (weak additivity of ReLU and commutativity) are identified as sufficient for LLFC, and experimental evidence is presented for these conditions occuring in LMC networks.
Strengths
The paper unifies two lines of research into LMC (spawning versus permutation methods) under a common mathematical framework. The notion of commutativity is an interesting derivation and not as obvious as it seems at first glance. The paper is well written and easy to follow, with clear definitions. If established comprehensively, LLFC would be a significantly stronger condition than LMC, and would have far-reaching implications.
Weaknesses
Main issues: - Current alignment algorithms for permuting LMC networks already directly optimize for LLFC (section 5.3), so the finding that LLFC implies LMC (Lemma 1) is formalizing a well-established phenomenon which is somewhat obvious. - As a result, LLFC implying LMC is somehow less interesting than LMC implying LLFC, but the latter direction is not explored. Only co-occurence is observed experimentally, making the direction of causation unclear. In particular, if LMC is found to imply LLFC, the reasons are likely to be extremely informative. - For spawning LMC networks, it is unclear how the early training epochs contribute to enabling LMC. The experiments do not consider the evolution of LMC networks through training time. Furthermore, the connection between figure 6 and the equation in lines 285-286 is unclear (see questions below). Additionally, the experimental evidence lacks baselines for comparison. In general, a fair comparison should include interpolated non-LMC networks. - Figure 2 and 3: no cosine similarity for interpolated networks that are not linearly connected. Since interpolating between any vectors (including random ones) increases their cosine similarity considerably, it is not clear that the increase in similarity is due to LLFC and not averaging. - Figure 4: no sparsity baseline for random networks. Currently it is unclear what is considered "small" in line 232. - Figure 5: no commutativity distance for non-LMC networks
Questions
- Section 5.2: how does similarity of $U_A$ and $U_B$ imply commutativity when $W = U \Sigma V$ also depends on $V$, which could differ significantly between two networks? The connection between figure 6 and the equation in lines 285-286 is unclear. - How much would solving the quadratic assignment problem improve over existing algorithms? Given that activation similarity is closely correlated with weight similarity, it is not obvious whether minimizing the QAP will lead to significant improvements in permutation alignment versus aligning weights and/or alignments, which is equivalent to minimizing one or both sides of equation 9 separately. - Weak additivity seems to rely heavily on the ReLU function. Does LLFC occur for other activation functions? - Do networks exist where LMC holds but not LLFC? If so, how can one find such networks and how common are they?
Rating
5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
3 good
Presentation
4 excellent
Contribution
2 fair
Limitations
Overall the findings could be explored in greater depth and better related to observable phenomena. While LLFC is shown to imply LMC, the reverse is not shown or refuted. For spawned LMC networks, it is not clear whether LLFC is a byproduct of averaging, how LLFC relates to the onset of LMC at an early point in training, or how LLFC can be increased/decreased. For permuted LMC networks, current alignment algorithms (e.g. Ainsworth et al. 2022, Singh & Jaggi 2020) are already explicitly optimizing for LLFC (via weight or activation alignment), and thus the findings have limited explanatory power.
Summary
This paper introduces Layerwise Linear Feature Connectivity (LLFC). Compared to the better known linear mode connectivity (LMC), which states that networks trained by SGD are linearly connected modulo permutation, LLFC suggests that the feature maps of every layer is connected. As shown in the paper, LLFC is a strictly stronger property than LMC. Empirical results on a range of architectures (ResNet-20, VGG-16, MLP) and datasets (MNIST, CIFAR-10) suggest that LLFC co-occurs with LMC. The authors then identify two conditions that collectively imply LLFC: weak additivity, which requires ReLU to behave like a linear activation on two modes, and commutativity, which requires the next-layer linear transformations applied to the internal features of two networks can be interchanged. They verify that these two properties holds for modes that satisfy LLFC empirically. Finally, the authors show that two common methods to obtain linearly connected modes, the spawning method and the permutation method, both promote the commutativity property, which explains their effectiveness.
Strengths
- The paper reveals a novel and more general notion of linear mode connectivity, which is an interesting phenomenon that attracts recent attention in the ML community. The authors provide precise definitions and a set of sufficient conditions for LLFC. Their observation is novel and advances the understanding of the origin of linear connectivity. - Experiments are sound and provide useful insights. In particular, empirical results support the occurrence of LLFC and validates that the two conditions for LLFC approximately hold in common settings. - Through dissecting the conditions required for LLFC, this paper also explains why the spawning method and permutation method produces LMC. Since theoretical results are sparse in explaining the origin of LMC, this paper’s contribution on the topic is significant. - The writing is clear and well organized.
Weaknesses
- The analysis for the cause of LLFC is limited to ReLU activation. Extending the weak additivity condition to different activations does not seem straightforward. To demonstrate the prevalence of LLFC, it might help to include experiments on neural networks with different activations. - As pointed out by the authors and shown in figure 4 and 6, weak additivity for ReLU activations and commutativity only approximately hold in real neural networks. Hence Theorem 1 describes conditions for perfect LLFC in an idealistic setting. While this result is significant and novel, a more careful treatment of the approximated version could make the theoretical contribution stronger. - There are some gaps between theory and empirical observation. Specifically, the following results are not well explained: (a) Why are the pre-activations sparse? Is this universal to all architectures? (b) Why do the modes obtained by the spawning method share similar principal directions of model weights in each layer? While these are empirically verified in the paper, it would be helpful to point the readers to existing theoretical analysis, if available. - While LLFC is an interesting observation and provides insights on LMC, there seems to be few applications that leverages LLFC. The idea of averaging features mentioned in the conclusion is interesting, but it is not clear how to implement feature averaging and under what situations this would be beneficial.
Questions
- In the definition of LLFC, there is a scaling factor $c$ that is not predicted by theorem 1. According to the authors, this inconsistency can be attributed to the accumulation of errors in the two conditions (line 263). Why does the accumulation of errors result in a scaling difference instead of another type of modification such as an additive term? - Does spawning or permutation have any impact on the sparsity of pre-activation $\tilde{H}^{(l)}$ and the weak additivity condition? - Is condition 1+2 the only way to guarantee LLFC? In particular, identifying other sets of sufficient conditions for LLFC may lead to new permutation methods that find linearly connected modes, other than weight matching or activation matching. - The experiments use SGD with momentum in training. Does the choice of optimization method affect the occurrence of LLFC?
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
3 good
Presentation
3 good
Contribution
3 good
Limitations
The authors adequately addressed the limitations. There are no potential negative societal impacts of this work.
Summary
In this work, the property of Layerwise Linear Feature Connectivity (LLFC) of neural network representations is introduced, which is a stronger generalization of linear mode connectivity (LMC). They show that LLFC often occurs when LMC does. Moreover, they give a possible mechanism by which LLFC may occur (ReLU activation additivity and weight commutatitivty), and provide evidence that LLFC often does occur by this mechanism. Finally, they reinterpret spawning and permutation-finding based methods for LMC as promoting commutativity.
Strengths
1. Introduction exposition is quite good. 2. The relationship between spawning / permutation-finding methods and commutatitivity is really interesting and insightful. The fact that both the weight-matching and activation-matching losses can be found in the commutativity condition is quite neat, as (to the best of my knowledge) these two methods were proposed as heuristics. The discussion of weight matrix rank in Appendix E is also insightful. 3. Figure 5 on the empirical measurement of commutativity is very nice, especially because the two baseline numbers (weight distance and activation distance) are so large, so commutatitivity really seems to be a good thing to look at here. 4. Although LLFC is stronger than LMC, in some sense it may make the study of LMC easier, because you have more to look at (features in each layer and the two sufficient conditions you give, not just loss of a whole neural net).
Weaknesses
1. You don't really cover the last method (straight-through estimator) from Git Re-Basin, even though it works the best in many cases. Perhaps you should note this. 2. Although these experimental setups are standard, it would be good to see to what extent this holds beyond these few architectures for image classification.
Questions
Typos: 1. In 5.2, (i) should have $W_{B, \mathrm{pri}}^{(l)} H^{(l-1)}_A + W_{A, \mathrm{pri}}^{(l)} H_B^{(l-1)}$. 2. 5.3: "conprehensive" -> "comprehensive"
Rating
8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
4 excellent
Presentation
4 excellent
Contribution
4 excellent
Limitations
I do not see much discussion of limitations, besides on the hardness of exactly minimizing the commutativity property objective. Perhaps you could state that the empirical evidence is limited to image classification.
Summary
This paper presents a special case of Linear Mode Connectivity (LMC) denoted as Layerwise Linear Feature Connectivity (LLFC). Whereas two trained neural networks present LMC if a convex combination of their parameters produce a neural network with similar training loss and accuracy, two trained neural networks present LLFC if a convex combination of their features in every layer also produces a neural network with similar features. Although being a special case, the authors find out that LMC and LLFC co-occur very often. Moreover, the authors characterize conditions that jointly imply LLFC between two ReLU networks. ***** Following the authors' response, I am updating my score from 6 to 7.
Strengths
The writing of the paper is very clear and the experiments are distributed in such a way along the text that it is easy to follow along. Moreover, I find the study of how this conditions may emerge in ReLU networks particularly relevant, since in that simpler setting it is easier to understand what is going on.
Weaknesses
The authors sell the idea of LLFC in a very positive tone, which is actually quite common in papers, but I cannot help but wonder about the following: Two neural networks presenting LLFC would have very similar parameters, and the fact that LLFC co-occurs with LMC implies that LMC is observed due to the similarity between trained neural networks - either because the first epochs of training determined what model would be ultimately obtained; or because there are relatively few optimal neural networks upon permutation. However, even under this interpretation, I believe that this study helps demystifying LMC if it turns out that LMC rarely occur without LLFC.
Questions
The introduction attributes the first observation of LMC to [8], but Section 2 attributes it to earlier work [21]. Definition 2 needs rework because it is clearly invalid if $\alpha=0$ or $\alpha=1$ unless $c=1$. In fact, this is not even how you measure it in Section 4, since you use cosine similarity instead. I believe that the use of a tolerance factor $\epsilon$ like you did in Line 220 would be more adequate in this definition. Would it make sense to assume that Definition 3 has some connection with stable neurons in ReLU networks, as in the following papers: https://arxiv.org/abs/1711.07356 & https://arxiv.org/abs/2102.07804 ? It seems to me that Condition 1 was inspired by spawning and Condition 2 by permutation, even though they are not worded in that way. Is that a valid way to interpret them? I would like to hear the thoughts of the authors about what I described in Weaknesses.
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
3 good
Contribution
3 good
Limitations
I could not identify a discussion about limitations in the paper.
Thank you for the extensive revisions to the figures to include convincing baselines. I have increased my score accordingly. Despite the straightforward connection between weight/activation matching and LLFC, I concede that LLFC is a useful formalization of these heuristics. However, given that permutation methods are still basically directly optimizing for LLFC, I am still skeptical of the insights gained into LMC by this work. In particular, I would appreciate a more thorough exploration of how the theoretical findings relate to practical applications in the following points. 1) Weak additivity: given that you already have preliminary results on different activation functions, I would have liked to see a comparison of the LMC of networks relative to the degree to which their activation functions are weakly additive. The point is not just to use widely-adopted activation functions, but to experimentally verify that weak additivity is an important condition for LLFC and/or LMC. For example, if it were possible to get LMC with non-ReLU activations despite not having LLFC, this would be extremely informative on the reverse direction (LMC implying/not-implying LLFC). 2) Minimizing QAP: the point is not to solve QAP, but to ask how much of an advantage QAP has only solving the LAP for activations or weights, since in the limit (infinite width or networks that are exact permutations of one another) all of these are equivalent. In particular, the positive terms of the QAP can simply be solved as a bigger LAP, so the only question remaining is how large the negative cross terms are relatively speaking. But there may be reasons for these cross terms to be small, hence guaranteeing the effectiveness of the heuristics - e.g. conceptually weak additivity and commutativity seem to be findings in a similar vein. Alternatively, maybe the cross terms are significant and reveal where the heuristics fail. I would greatly appreciate more discussion of these cross terms in the paper.
Thank you for appreciating our contributions and for raising the score! **Weak Additivity with other activations** Thank you for the suggestion. We did some preliminary experiments on different activation functions. Specifically, we conduct experiments on MLPs on MNIST with different activation functions, including ReLU, Sigmoid, and Tanh. For each model with different activation functions, we apply the spawning method to obtain two modes, $\boldsymbol{\theta_A}$ and $\boldsymbol{\theta_B}$, spawning at the same epoch. Over the test set $\mathcal{D}$, we measure the error barrier (defined in Frankle et al. [8]) between $\boldsymbol{\theta_A}$ and $\boldsymbol{\theta_B}$, and measure the LLFC and the weak additivity condition. Here, error barrier is denoted as $\text{Err}\_{Barrier}$, where $\text{Err}\_{Barrier} = \max\_{\alpha \in [0, 1]} \text{Err}\_{\mathcal{D}}(\alpha \boldsymbol{\theta}_A + (1-\alpha) \boldsymbol{\theta}_B) - \frac{1}{2}(\text{Err}\_{\mathcal{D}}(\boldsymbol{\theta}_A) + \text{Err}\_{\mathcal{D}}(\boldsymbol{\theta}_B))$. In the table below, for the cases of MLP(ReLU) and MLP(Sigmoid), the values of both $\text{Err}\_{Barrier}$ and $\mathbb{E}\_{\mathcal{D}, \ell \in [L]}[1-\text{cosine}^{(\ell)}\_{0.5}(\boldsymbol{x}_i)]$ are close to zero; for the case of MLP(Tanh), those values are not negligible. Correspondingly, the values of $\mathbb{E}\_{\mathcal{D}, \ell \in [L]}[\text{Dist}\_{\sigma}(\tilde{\boldsymbol{h}}\_{i, A}^{(\ell)},\tilde{\boldsymbol{h}}\_{i, B}^{(\ell)})]$ for MLP(ReLU) and MLP(Sigmoid) are relatively small compared to MLP(Tanh). This observation demonstrates a correlation between the onset of LMC/LLFC and the weak additivity condition, thus suggesting that weak additivity condition is important for LMC/LLFC. Also, we did not find any instance that LMC holds but LLFC doesn't. | | $\text{Err}\_{Barrier}$ (%) | $\mathbb{E}\_{\mathcal{D}, \ell \in [L]}[1-\text{cosine}^{(\ell)}_{0.5}(\boldsymbol{x}_i)]$ | $\mathbb{E}\_{\mathcal{D}, \ell \in [L]}[\text{Dist}\_{\sigma}(\tilde{\boldsymbol{h}}\_{i, A}^{(\ell)},\tilde{\boldsymbol{h}}\_{i, B}^{(\ell)})]$ | | ------------ | -------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ | | MLP(ReLU) | 0.115 | 0.0385 | 0.1525 | | MLP(Sigmoid) | 0.060 | 0.0203 | 0.0799 | | MLP(Tanh) | **2.910** | **0.1157** | **0.4058** | **Minimizing QAP:…ask how much of an advantage QAP has only solving the LAP for activations or weights…** Thank you for the very interesting comments. We agree that further studying the advantages of solving the QAP over LAP is an important future direction and will make sure to discuss these points in the paper.
I appreciate and am satisfied with the responses from the authors. I am updating my score as a reflection of that.
Thank you for the clarification and revisions to the result as I suggest. I believe that LLFC is an interesting phenomenon and will be helpful to many researchers.
Thank you for the detailed reply. I have read the rebuttal and other reviews and will maintain my score.
Thank you for the rebuttal
Dear authors, thank you for providing a rebuttal, I will take it into account when giving my recommendation. Best, Your AC
Decision
Accept (poster)