Cascade of phase transitions in the training of Energy-based models

In this paper, we investigate the feature encoding process in a prototypical energy-based generative model, the restricted Boltzmann machine (RBM). We start with an analytical investigation using simplified architectures and data structures, and end with numerical analysis of real trainings on real datasets. Our study tracks the evolution of the model’s weight matrix through its singular value decomposition, revealing a series of phase transitions associated to a progressive learning of the principal modes of the empirical probability distribution. The model first learns the center of mass of the modes and then progressively resolve all modes through a cascade of phase transitions. We first describe this process analytically in a controlled setup that allows us to study analytically the training dynamics. We then validate our theoretical results by training the binary–binary RBM on real datasets. By using datasets of increasing dimension, we show that learning indeed leads to sharp phase transitions in the high-dimensional limit. Moreover, we propose and test a mean-field finite-size scaling hypothesis. This shows that the first phase transition is in the same universality class of the one we studied analytically, and which is reminiscent of the mean-field paramagnetic-to-ferromagnetic phase transition.

Paper

References (52)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer vpPo6/10 · confidence 4/52024-07-10

Summary

This paper analytically demonstrates a cascade of second-order phase transitions in the training of a simple RBM model and numerically validates this theory in the training on real datasets.

Strengths

- This paper theoretically analyzes the learning dynamics of weight parameters in the Binary (or Bernoulli)-Gaussian RBM (BG-RBM) through the lens of statistical physics. - The authors demonstrate that the RBM undergoes a series of second-order phase transitions during learning, both theoretically in the BG-RBM and numerically in the Binary-Bernoulli RBM (BB-RBM).

Weaknesses

- There is a gap between the theoretical and numerical analysis in this paper. Specifically, the BG-RBM with a single hidden node is utilized in the theoretical analysis in Section 4, while the BB-RBM is used in the numerical analysis in Section 5. - Although the main results of this paper may merit acceptance into NeurIPS, I believe the authors should revise the manuscript to improve readability and clarity. Some suggestions are provided in the Questions section below. If the authors address these issues and answer my questions, I can strongly recommend accepting this paper for NeurIPS; however, I cannot recommend it for acceptance in its current form.

Questions

Major - While the authors consider a single Gaussian hidden node, they do not provide any statistical details of the Gaussian distribution, such as the mean and variance. I assume the authors consider a Gaussian distribution with zero mean and $1/N$ variance, but I am uncertain about the meaning of $N$ (I guess $N \equiv N_v$), why the authors choose this setup, and how $N$ relates to the statistics of $xi_i$. - The meaning of the equation between lines 161 and 162 is unclear, especially when the authors mention that the equation “is analogous to the one studied in the previous section.” - While the authors analytically investigate the BG-RBM with a single hidden node in Section 4, the numerical analysis in Section 5 is conducted on the BB-RBM with multiple hidden nodes. The authors should discuss the expected differences due to this gap and adequately clarify the limitations of their analysis. - I am not sure why “the projections along all subsequent directions are Gaussian” as stated in line 231 on page 6. - It is unclear how the authors obtained Figure 4 E and what they aim to demonstrate. Does the $h$ in Fig. 4 E indicate the hidden variable? - The direct relationship between Section 4 and Section 5 is hard to capture. As I understand it, the main part of this paper introduces an underlying preferred direction $\xi_i$, but I am not sure which vectors correspond to $\xi_i$ in Sec. 5. Please describe the correspondence between Section 4 and Section 5 and clarify the differences from the results in Ref. [8]. Minor - While the authors mention the Mattis model to describe $\xi$, the explanation of $\xi$ is insufficient. It is unclear whether $\xi$ is frozen, the distribution of $\xi$, etc. - In Fig. 1, the x-labels of the left and right panels differ. The term "susceptibility" is not defined, each line is indistinguishable, and the meanings of the left and right vertical axes are not indicated in the inset of the left panel. There is no indication of the black solid lines in the insets of the right panel. - In lines 127 and 439, the condition for diverging the correlation seems to be a typo. $N^{-1}\sum_{i} w_i^2 \approx 1$ may be correct. - Please introduce the learning rate $\epsilon$ in Eq. (3) because the authors provide results with respect to the learning rate in Fig. 1. - Please provide references and detailed descriptions for the BG-RBM primarily considered in Section 4. - The authors should correct numerous typos in the main text and the appendix. For instance, the authors often use expressions such as $s_i$ and $N$, which are not defined in this paper. In line 107 on page 3 and line 416 on page 12, there are typos in the definition of magnetization $m$. In line 414 on page 12, there is a typo in the definition of $\xi_i$. In Eq. (3), the authors do not clarify what $t$ indicates. - The authors should consistently use expressions throughout the paper. In lines 136 and 194, the word "appendix" is used, but in other cases, the term "SI" is used. The title of Appendix B is "Binary-Binary RBM," but the model is often referred to as "Bernoulli-Bernoulli RBM" in the paper.

Rating

6

Confidence

4

Soundness

3

Presentation

1

Contribution

3

Limitations

The authors should discuss the limitations of their work more thoroughly.

Authorsrebuttal2024-08-05

Detailed answers to the questions (part 1)

**Detailed anwers to the questions:** 1. The referee is right, we should have been more careful when defining the quantity. The distribution of the hidden nodes is indeed of zero mean and variance of order $1/N_v$. The value is indeed $N=N_v$. This is the correct scaling to have a large dimensional (large $N_v$) limit. Let us clarify better this point. In a general formulation, we can consider that the probability distribution of the hidden nodes follows a Gaussian distribution with a variance $\sigma_h^2$. In this case, the effective model on the visible nodes is given by the following Hamiltonian $H=-\sigma_h^2 ( \sum_i s_i w_i )^2$, where we assume that the weights are of order $\sim O(1)$. This model can be analyzed as a function of \sigma_h. In practice, if $\sigma_h^2$ scales as $N^\alpha$, where $\alpha<-1$, the system is dominated by its entropy and remains in the paramagnetic phase where the average of each variable $\langle s_i \rangle = 0$ since the interaction is too weak. If \alpha>-1 the interaction is very strong and the system is frozen in configurations where $\langle w_i s_i \rangle = \pm 1$. Therefore the correct scaling is when the weights are of order $\sim O(1)$ and the variance of the hidden $\sim O(1/N_v)$ in order to observe a transition (as the magnitude of the weights change) of a disordered phase without particular orientation, and an ordered phase where the variables are parallel to $w_i$. For practical experiments, it is not necessary to impose the scaling since the learning process bring the system into the interesting regime where information is stored in the model parameters, as illustrated by our experiments. 2. The sentence at lines 161-162, refers to the type of phase transition occurring in the system. In the case with multiple features, the first phase transition is very similar to the one with one feature, in the sense that it is of the same “nature”. In this regime, corresponding to the first phase transition, everything happens as if there were just one main direction, which is $\xi^1 + \xi^2$. We will correct this sentence to make it clearer in the final version. 3. The theory is written for the binary-Gaussian RBM in the main text and for the binary-binary RBM in the Appendix. In the latter case, we show that in order to observe an interesting behavior we need to have an extensive (large) number of hidden nodes. On this aspect the gap is not strong. The main difference between theoretical and experimental parts relies on the dataset which is much simpler in the former case. Based on the article [8], we expect that even if the dataset present various modes (small compared to the number of visible/hidden nods), the system can still be described in a mean-field setting and, consequently, the phenomenology observed for one or two modes can be extended to many modes (but only a small number). This is confirmed by our experiments. Now we do not claim that this description is valid all along the training process, and we can see that at large training time the mean-field theory description is not as accurate anymore. We will comment on this in the final version. 4. This sentence might be indeed unclear and thus will be rewritten in the final version. By this sentence we intended to clarify that as the system is learning the first direction to split the data, the dataset projected on the remaining direction is mostly Gaussian distributed. The reason is that in the other direction, the projection is essentially given by the sum of random i.i.d. weights (non sparse) multiplied by the spins $s_i$. Furthermore, we can provide in the Appendix, if the reviewer thinks it would be useful, a figure showing that the projections on these directions corresponds to random noise as long as the other directions are not learned.

Authorsrebuttal2024-08-05

Detailed answers to the questions (part 2)

5. We thank the referee for pointing out that we should clarify further the panel E of Fig. 4. The aim of this figure was to show the “hysteresis” phenomenon which is a clear signature, devised and observed in statistical physics, to unveil that a high-dimensional probability measure had a phase transition at which it split in two different lumps. The procedure consists in tilting the probability measure by introducing in the energy function a contribution that favors one lump over the other. If the measure is indeed concentrated on two distinct lumps, one induces a sudden transition (“first-order transition” in physics). In our case, the lumps are associated to the learned patterns and this extra contribution consists in the scalar product between the visible variables and the learned patterns times $h$ (the field controlling the strength of the tilting). In the presence of a first-order phase transition, one generically finds the phenomenon of hysteresis, i.e. the transition from one lump to the other can be retarded because of metastability, thus leading to the characteristic hysteresis loops that we indeed show in Fig.4 E (see e.g. den Hollander, Frank. "Metastability under stochastic dynamics." Stochastic Processes and their Applications 114.1 (2004): 1-26; Bovier, Anton. "Metastability." Methods of contemporary mathematical statistical physics 1970 (1970): 177-221 for rigorous treatment and Chaikin, Paul M., Tom C. Lubensky, and Thomas A. Witten. Principles of condensed matter physics. Vol. 10. Cambridge: Cambridge university press, 1995 for a physics treatment). This figure therefore gives direct evidence of the decomposition of the measure in distinct lumps corresponding to the learned patterns, and that this decomposition takes place at the second-order phase transition happening during learning. We will explain better these points in the revised version. 6. To start with ref[8], this work presents a mean-field theory describing theoretically the static behavior of RBMs assuming a proper analytical form for the weight matrix. In this work, the phase transition for such a setting is derived but only the static behavior is analyzed theoretically. Our present work instead characterizes the dynamical behavior of the RBM. It shows at which rate the weight matrix is shaped by the dynamics, it describes the different mechanisms (evolution of the weights into a preferred direction, by which time evolution, etc.) in details, and we further show that this is what happen when performing a training in this regime. Also, we show how the weights evolve in the case where the clusters of the dataset are correlated. Concerning the relationship between sec 4 and 5, the reviewer’s comments helped us to understand that we have to clarify the relationship between the SVD decomposition used in Sec. 5 with the projections used in Sec. 4. There is a direct connection between the PCA and the preferred directions ($\xi^1 + \xi^2$ and $\xi^1 -\xi^2$) used in the theoretical analysis, but it was not clear enough in the current version. You could see it from the appendix, but the point was not made explicitly anywhere. We will discuss this and make it explicit in the revised version. In fact, by looking in the appendix at the eq. after line 491, we decomposed in our model the correlation matrix on the vectors $\eta$ and $\bar{\eta}$: this is the SVD decomposition for this very simple model. It is therefore clear that since the model exhibits a growth first towards \eta, the first eigenvectors of $\langle s_i s_j \rangle$ and then in the direction of $\bar{\eta}$, the second direction, we are indeed proving that the dynamics is driven by the PCA of the dataset. We think that the part on the divergence of susceptibility is the clearest link between the two sections. Section 4 shows that the analytical behavior is confirmed in numerical experiments with real data sets: first, the relation between the PCA and the SVD of the RBM weight matrix, second, the phase transition: how the susceptibility diverges (as the system size increases and with a known exponent) as well as the divergence of the mixing time. We will make clearer in the final version the link between the two parts.

Reviewer vpPo2024-08-11

Thank you for the authors' careful responses and explanations. After reading the responses, I have resolved most of my concerns regarding the paper. However, I remain uncertain about the readability of the manuscript, as I am unable to review the revised version, which is expected to differ significantly from the current draft. I understand that this limitation is due to the conference's policy, and while the authors may be disappointed, it makes it difficult for me to increase the presentation score. Nonetheless, I acknowledge the significance of this work and have updated my score accordingly (4 -> 6). I hope the authors will take care to clarify the notations and statements, and include the limitations of this work in the final version.

Reviewer ZU376/10 · confidence 3/52024-07-12

Summary

This paper presents an analysis of the phase transitions of RBMs through an analysis of the weight matrix. They find the dynamics tend toward the center, then diversity into modes. The theoretical results are supported by empirics on 3 ML datasets.

Strengths

- This expands the important field of exploring the dynamics of energy based models - The figures are generally well presented (conditioned on note below) - The authors offer a compelling argument for the scalability of these phase transitions, and the pipeline theory to CelebA is well done

Weaknesses

- This paper would be right at home in a physical review journal, but given the audience of NeurIPS, I would recommend increasing the accessibility to the very ML focused audience (e.g. matching the notation used in ML world, things like labeling axes with descriptive words then putting the equations in text, numbering all equations, etc.) - 199 paragraph is too long

Questions

- How do you imagine these transitions changing with respect to sampling convergence? Perhaps out of scope of this work, but would be interesting to see all of these plots generate but now with x axis being sample steps and see how that looks - What would be the biggest takeaways/generalization to be made from RBM phase transitions to more complex ML models?

Rating

6

Confidence

3

Soundness

2

Presentation

3

Contribution

2

Limitations

The authors sufficiently addressed societal limitations.

Reviewer mUPE7/10 · confidence 3/52024-07-12

Summary

This paper analyses the phenomenon of phase transitions in the training dynamics of restricted Boltzmann machines (RBM) in theory and practice. In the theory part, the phase transition is characterised for a toy model: The RBM with a hidden node per feature is fitted to data characterised by one feature vector $\xi$ or two correlated feature vectors $\xi_1, \xi_2$. The phase transitions are detected as diverging correlation functions between visible states of the RBM and via exponential fitting of the RBM weights to the target features. In the empirical part, the phase transitions are explored in the training of a RBM on the Human Genome Dataset, MNIST, and CelebA. It is observed that the singular values of the weight matrix and the susceptibility undergo sudden changes during training, indicating a phase transition.

Strengths

- The theoretical evaluation seems sound and in general the paper is well written. - The paper explores an interesting phenomenon in training dynamics of RBMs. The experimental evaluation of the phase transition is compelling and highlights the emergence of phase transitions well. Similar studies may be applicable to other types of generative models.

Weaknesses

- It seemed to me that the notation switches between $v$ and $s$ to denote visible states/spins/sites. - I think the paper could be improved by making a stronger connection between theory and empirical evaluation. For example, do the projections and projected weights in the theoretical section correspond exactly to the discussed singular value decomposition described in the empirical evaluation? Can the susceptibility be analysed in the theoretical section? More consistent nomenclature of the terms described would help with reading and understanding the paper. - The numerical analysis would benefit from paragraphs or subsections. - The analysed dynamics of the RBM weights don't fully reflect the actual training dynamics. In practice, RBM training requires self-sampling from the model which causes biases in the estimated negative term. I would expect that the sampler is adversely affected by the phase transitions in the model (e.g. sudden drop in mixing time). Considering that sampling in an RBM is relatively tractable (compared to other types of EBM) I am wondering if the training dynamics of e.g. one step contrastive divergence can be analysed in a similar fashion. However, the authors do address the effect of the phase transition on the MCMC sampling in the experimental section.

Questions

- Do the projections and projected weights in the theoretical section correspond exactly to the discussed singular value decomposition described in the empirical evaluation? Is a theoretical study of the susceptibility in the theoretical section feasible? - Can the phase transitions be detected during training to, for example, adjust for the longer thermalisation time in the vicinity of the phase transition? - When running a gradient based Langevin sampler during training, do your observations suggest spiking parameter updates in the vicinity of the phase transition due to second order discontinuities, thus creating potential training instabilities?

Rating

7

Confidence

3

Soundness

4

Presentation

3

Contribution

3

Limitations

The work analyses the phenomenon of phase transitions in RBM training dynamics, and is limited to this setting by definition. The scope is clearly defined in the paper. However, the authors checked the limitations box in the check list without providing further justification in the main text or in the checklist. A brief sentence about the scope of the work or a reference to lines in the main text would be appropriate.

Reviewer pfhu6/10 · confidence 3/52024-07-13

Summary

This work examines the relation between the internal representations of an RBM and the data that it is trained on as the training procedure goes forward. The work begins with a theoretical exposition for Binary-Gaussian RBMs trained CW-like models with 1 or 2 preferred encoding patterns. The theoretical analysis shows that the weights develop along directions that aggregate the directions of data modes before further settling into directions that capture each of the data modes. The analysis is extended to Binary-Binary RBMs. Several empirical results are presented. For simple models, theoretical findings are numerically validated. RBMs are also learned for HGD, MNIST, and CelebA dataset. The behavior of these learning situations mirrors certain behaviors of the simple cases. The RBM first learns patterns along the largest principal components of the data before undergoing a phase transition and settling into more nuanced representations.

Strengths

* The dynamics of generative model training and the relations between the data and internal representations of generative models is an important area of study. I appreciate that this work undertakes the challenging task of unraveling these behaviors. * This work offers interesting evidence that, across simple and more complex datasets, RBM learning has somewhat predictable behavior. In particular, RBM weights will first learn to represent the data directions with the highest variation before learning more refined representations that better capture modal structure. * Extensive derivations and empirical results are included.

Weaknesses

* A major weakness of this work is the scope. While broad claims are made about implications for other generative models, the analysis presented seems specific to the RBM architecture. Studying more recent generative models with stronger capacity would greatly increase the impact of this work, although such analysis is very difficult. * The work is quite dense, and it can often be difficult to follow the lines of reasoning and conclusions based on derivations. While the topic is complex and this is to some extent inevitable, I recommend the authors make efforts where possible to streamline and clarify the presentation.

Questions

Are there conclusions from this work which can be extended to more powerful generative models such as diffusion models, deep EBMs, etc.?

Rating

6

Confidence

3

Soundness

3

Presentation

2

Contribution

3

Limitations

Limitations are not discussed in detail.

Reviewer ZU372024-08-08

I appreciate the authors response, and the changes will improve the quality of the paper, as such I have updated my score to reflect this (5 -> 6).

Reviewer mUPE2024-08-13

I thank the authors for their helpful response to my questions. I understand that analyzing CD-1 is more challenging, and I am satisfied with how the authors have addressed my concerns. I believe the conclusions of the paper remain valid; however, I would suggest to discuss the non-equilibrium nature of RBM training in the paper. With the suggested improvements in the exposition, I consider this work to be a valuable contribution to the understanding of restricted Boltzmann machines from a statistical physics perspective. I would like to raise my score to 7 and recommend this paper for acceptance.

Reviewer pfhu2024-08-14

Thanks for the response. I will keep my score.

After reading the author responses to myself and other reviewers, I maintain my current score and raise my contribution score. My main recommendation for future versions of the paper is to make the text as clear and readable as possible.

© 2026 NYSGPT2525 LLC