Only Strict Saddles in the Energy Landscape of Predictive Coding Networks?

Summary

This paper studies the properties of the loss function of predictive coding (PC) networks. In PC networks, the loss function that is optimized is not a typical loss such as the MSE, but the “equilibrated energy” (or “PC energy”). The paper argues that the PC energy has some better theoretical properties than the MSE loss. Specifically, the paper conjectures that all saddle points of the PC energy are strict, in contrast with the MSE loss that has non strict saddle points. To support this conjecture, the paper compares the properties of the MSE loss function and the PC energy function around the point theta=0 in deep linear networks. In particular, they compare the spectral properties of the Hessian of these functions at theta=0. They also prove that non-strict saddle points of the MSE loss (other than theta=0) become strict in the PC energy. The paper also provides some empirical evidence that PC escapes the saddle point theta=0 of the PC energy faster than BP escapes the saddle point theta=0 of the MSE loss, even in nonlinear networks.

Strengths

The paper is clearly presented. The topic is important (PCNs and related energy-based networks deserve attention as we are looking for alternative, more energy-efficient, computing paradigms.) The question raised is quite interesting. Even though there is a lot of extrapolation, the argumentation to support the conjecture is well presented and sound, too. Although the paper focuses specifically on PC networks that employ full clamping, my impression is that the analysis can likely be extended to other settings as well, with different energy functions (i.e. different “energy-based networks”), and alternative, more effective ways to do training (e.g. by nudging the output, like in Equilibrium Propagation, instead of fully clamping the output). See my questions below.

Weaknesses

The theory is restricted to linear networks. Simulations are performed only around the saddle point theta=0.

Questions

1. Line 25: “PC uses only local information available to each neuron to update weights…” Is it really the case though? To my understanding, it feels like PC needs both the transpose of the weights (in the feedback path) and knowledge of the derivative of the activation function (in nonlinear networks). It might be a very basic question, but I would like to understand what problems of BP are solved by PC networks from a hardware (neuromorphic) perspective. Could you clarify this point? 2. Is there a consensus in the deep learning literature that the strict/non-strict saddle point issue is what determines whether an optimization problem is easy/hard to solve? (It is fine if there is no consensus, it is a reasonable assumption and this question should be studied anyway). 3. Is it readily clear that the point theta=0 is a critical point of both the MSE loss and the PC energy? Is there a straightforward argument to see that the gradients of these functions at theta=0 vanish? 4. It is shown that theta=0 is a non-strict saddle point of the MSE loss. Is this also true for other loss functions? (e.g. the cross-entropy) 5. In the literature on Predictive Coding in general, it seems to be standard practice to clamp the outputs to the desired outputs. Is there a fundamental reason for doing that? Ref [1] below shows that pushing the outputs away from desired outputs (“negative nudging”) actually works significantly better than pulling them towards desired outputs (“positive nudging”) or clamping them. Note: the simulations of Ref [1] are performed on deep convolutional Hopfield networks, not PC networks, but the theory applies to PC networks too. 6. Related to Question 5 above, can the theoretical analysis performed in this work be directly transferred to the case when using negative nudging as in Ref [1] ? Or does it only work when clamping outputs to desired output values? 7. Could the analysis of the present paper be extended to other energy-based networks (not necessarily PC networks) ? Ref [2] below studies the effect of learning via Equilibrium Propagation on the Hessian of (linear) energy-based physical networks, which looks (at least superficially) similar to the theoretical study of this paper. References [1] Scellier, Benjamin, et al. "Energy-based learning algorithms for analog computing: a comparative study." Advances in Neural Information Processing Systems 36 (2024). [2] Stern, Menachem, Andrea J. Liu, and Vijay Balasubramanian. "Physical effects of learning." Physical Review E 109.2 (2024): 024311.

Rating

7

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

The paper focuses on a specific feature of loss functions, namely the presence/absence of strict/non-strict saddle points. I would expect, however, that there are more important features than this one to effectively train PC networks. See my questions above.

Summary

This paper explores how the energy landscape in deep linear networks trained with predictive coding compares to the loss landscape of DLNs trained with backpropogation. The authors prove that the energy of PC is a rescaled version of the MSE loss. The authors then show that, unlike BP trained DLNs, multiple points in the energy landscape of strict saddles. This suggests a way in which training with PC can escape poor initializations faster than BP. To support these theoretical results, the authors provide numerical results.

Strengths

1. The paper was well written and well motivated. It was clear what the problem the authors were tackling was, why it was important, and what the main results of the paper were. 2. The combination of theoretical and numerical results that collectively contributed to a strong, compelling story was great.

Weaknesses

Major C1: Given that the authors prove that, for DLNs, the equilibriated energy is a scaled version of the MSE, I believe an alternative explanation for why training with PC is able to achieve good performance faster than BP is because it has a different effective learning rate. Is that assessment correct? If so, did the authors explore whether changing the learning rate for BP led to changes in learning speed in the DLN and DNN numerical examples? Major C2: While there is technical discussion of PC in Section 2, it was not clear how that maps to the idea of PC in neuroscience. More discussion, even in the Appendix, would be useful for those familiar with PC more from the neuroscience perspective. Minor C1: It was unclear to me why $\textbf{W}_L$ had to equal 0 for the equilibrated energy (line 235). I read the subsection in the Appendix that is referenced and still felt like the comment was a little abrupt. Minor C2: Given that the authors have some remaining space, it would be interesting and good to hear more about the neuroscience perspective, other than learning speed. Very minor C1: $\textbf{g}_f$ is used before it is defined (line 154). Very minor C2: Why is there a 1/2 in the MSE loss? To the best of my knowledge, this is not "standard" as is quoted in line 160.

Questions

1. Does PC still show an advantage over BP if a different learning rate is used for BP (so as to match the effective learning rate of PC)? 2. Why must $\textbf{W}_L =0$ for the saddles considered at the end of the paper? 3. Why would having strict saddles be useful in cognitive science?

Rating

7

Confidence

3

Soundness

3

Presentation

4

Contribution

3

Limitations

The authors did a nice job addressing their limitations.

Summary

This work looks at the (equilibrated) loss landscape of predictive coding networks, and analyzes the nature of saddle points therein --- especially in comparison to that obtained for backpropagation based deep linear networks with MSE loss. They find that several non-strict saddle points in the latter turn to strict saddle points (with positive and negative eigenvalues) in the former. This is argued to lead to better convergence properties, and is shown empirically for some simple networks.

Strengths

- Studying the loss landscapes for predictive coding is a very interesting direction, and could be useful to better understand the benefits imparted by it, and also circumvent any idiosyncratic problems that might arise. - An interesting theoretical analysis of the Hessian for predictive coding is carried out, by considering the equilibrated loss, which can also be useful for future theoretical works in this space. - Non-strict saddles at origin or zero-rank saddles are shown as strict saddles in predictive coding, which afford for better convergence when training starts near such a saddle as compared to that with usual deep nets where these are non-strict saddles. The experiments support the point and associated visualizations are quite interesting.

Weaknesses

Some of the claims might a bit exaggerated: In the conclusion, the authors write that PC networks have only strict saddles. But, I don't think there is anywhere in the paper it is shown that there are only two kinds of saddles: zero and zero-rank, both of which are shown as strict saddles. No other saddles are analyzed, or shown to be non-existent, and the zero-rank saddle is not too dissimilar to the zero saddle. Likewise, there are statements saying that they prove non-strict saddles other than origin to be zero in PC network setting, but I think they are just talking about the zero-rank saddles. Experiments are somewhat restricted: The experiments are primarily with simple MLPs. It is also unclear when the backprop'ed networks are trained if their convergence results are well-tuned. In other words, if they use GD vs SGD, or if the learning rates or batch sizes are optimal, adaptive gradient methods, and the kind. (PC doesn't have to outperform them, but it will be good to gauge how well it fares in context of all these things that practitioners would normally employ) Section 3.2 analysis is more or less a corollary of Singh et al, 2021: The authors only somewhat acknowledge this in the appendix, but I think this should be properly highlighted and attributed in the main text of the paper as well. Loss Landscape of PCNs could have been studied a bit better: The technical part is largely based on Achour et al and Singh et al, and some more aspects of the loss landscape could have been carried out. The paper has a good contribution, but I am unsure if this is sufficient compared to a normal NeurIPS paper of 9 pages. Others: - Some of the terminology is loose and confusing: Often in their paper the talk about saddles of BP in reference to that of MSE. I think this is quite confusing, as what they mean by MSE is networks trained with Backprop on MSE. A better acronym would be BP-MSE or BP, as otherwise it is quite vague. - Line 49, Phenomenon of vanishing gradients has the wrong reference. The right one is Bengio 1994 https://www.comp.hkbu.edu.hk/~markus/teaching/comp7650/tnn-94-gradient.pdf, while the currently mentioned one is more about curvature. - Could the authors better describe the steps from eqn 70 to 71 in proof of proposition 1, and where all things are under approximation and where equality? It's a bit loosely written at the moment.

Questions

See the above section

Rating

5

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

^^