Counter-Current Learning: A Biologically Plausible Dual Network Approach for Deep Learning

Summary

Inspired by the counter-current phenomenon observed in nature, the authors propose a counter-current learning (CCL) framework that decouples the input and backpropagation information in different circuits, overcoming many biological implausibility issues of backpropagation learning (BP).

Strengths

I am not an expert in target propagation, but the general idea presented in this work sounds novel and interesting to me. I believe it also offers some insights into nature-inspired learning. I appreciate the constructive figures that help me quickly grasp the core idea of this work.

Weaknesses

- In Table 1, the authors should compare their results with other biologically plausible methods, like forward-forward and deep softhebb, or justify their criteria in model selection. - A minor drawback is the lack of validation in complicated dataset like Imagenet.

Questions

- I am surprised that the performance of BP is worse than the proposed local learning method. Can the authors explain more about this gap? - In forward-forward [1], Hinton proposes a recurrent network architecture that inputs the label at the top layer. This architecture seems similar to what is used in this work. Can the authors elaborate on the differences? reference [1]. Hinton G. The forward-forward algorithm: Some preliminary investigations[J]. arXiv preprint arXiv:2212.13345, 2022.

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

I think the authors have discussed it well.

Summary

In this paper, the authors propose counter-current learning (CCL), a biologically plausible framework for credit assignment in neural networks.

Strengths

The research direction of having new learning algorithms inspired biologically seems relevant and interesting.

Weaknesses

- The authors discuss biological plausibility as a motivation, but do not discuss thoroughly how the proposed approach is biological plausibility. For instance, update-locking and non-frozen activities should be discussed in detail. - The comparison with the state of the art in the domain is missing: [1] Hinton, G. (2022). The Forward-Forward Algorithm: Some Preliminary Investigations. Technical report. [2] Dellaferrera, G., Kreiman, G., and Kreiman, G. (2022). Error-driven Input Modulation: Solving the Credit Assignment Problem without a Backward Pass. In Chaudhuri, K., Jegelka, S., Song, L., Szepesvari, C., Niu, G., and Sabato, S., editors, Proceedings of the 39th International Conference on Machine Learning, pages 4937–4955. PMLR. [3] Andreas Papachristodoulou, Christos Kyrkou, Stelios Timotheou, and Theocharis Theocharides. Convolutional channel-wise competitive learning for the forward-forward algorithm. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 14536–14544, 2024. - The results are sometimes counter intuitive. For example, the results of CCL vs BP for CIFAR100 in Table 3. Why is CCL working considerably better than BP? - The presentation and organization of the paper can be improved. For instance, Figure 2 can be presented at a higher abstraction level as pseudo-code

Questions

1. Could the author discuss the biological plausibility of their approach from weight transport, locality, freezing of neural activity, and update locking points of view? 2. Could the author provide some insight how their approach compares against or relates to the state of the art in the domain: FF, PEPITA, and CFSE? [1] Hinton, G. (2022). The Forward-Forward Algorithm: Some Preliminary Investigations. Technical report. [2] Dellaferrera, G., Kreiman, G., and Kreiman, G. (2022). Error-driven Input Modulation: Solving the Credit Assignment Problem without a Backward Pass. In Chaudhuri, K., Jegelka, S., Song, L., Szepesvari, C., Niu, G., and Sabato, S., editors, Proceedings of the 39th International Conference on Machine Learning, pages 4937–4955. PMLR. [3] Andreas Papachristodoulou, Christos Kyrkou, Stelios Timotheou, and Theocharis Theocharides. Convolutional channel-wise competitive learning for the forward-forward algorithm. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 14536–14544, 2024. 3. Why is CCL working considerably better than BP in Table 3 for CIFAR100?

Rating

4

Confidence

4

Soundness

2

Presentation

2

Contribution

2

Limitations

The limitation of the work has been briefly discussed.

Summary

In this work, the authors proposed the counter-current learning algorithm, a novel algorithm for biologically plausible training of feedforward neural networks. The learning rule is built upon the target-propagation algorithm and its variants. In these algorithms, the backward pathway is typically trained in a way that the target information propagated to each hidden layer provides a layer-wise target that nudges the forward pathway's output toward the true label. Instead, the authors jointly trained both feedforward and feedback weights projected to each layer, in a way that forward and backward pathways exhibit roughly the same activity at each layer. The authors applied this algorithm to various image recognition tasks (MNIST/Fashion-MNIST/CIFAR10/CIFAR100) and an autoencoder task and demonstrated that the proposed algorithm outperforms other variants of target-prop algorithms. The authors empirically analyzed the learning process of the algorithm by examining the development of hidden layer representations and through an ablation study.

Strengths

The proposed learning algorithm is simple yet novel. The key novelty is the training algorithm for the backward pathway, where, unlike previous target-prop algorithms, activity matching between the forward and feedback pathway at each layer is used as the objective. I was positively surprised by the algorithm's strong performance because this learning algorithm clearly deviates from the backprop of the global loss. This training approach also makes the algorithm distinct from previous related approaches where the backward weights were fixed instead (Frenkel et al., Frontiers 2021; Shibuya et al., AAAI 2023).

Weaknesses

The learning rules for hidden weights have trivial solutions, hence susceptible to representation collapse. The authors discussed their techniques for avoiding the collapse briefly in the supplement, but the implementation is not discussed in detail. Biological plausibility of the regularization mechanisms should also be discussed.

Questions

Considering that the algorithm is minimizing the difference between $a_l$ and $b_l$, I expect to see large diagonal components in Figure 6 after learning. However, there is no diagonal trace in the figure. Why is that the case? In addition, at t = 0, $b_1$ to target are already weakly aligned with the input, but $b_0$ is not aligned with the input. What is the origin of initial alignment and the absence of it in $b_0$? On a related point, I wonder if the authors checked whether the stop gradient is implemented correctly, for instance, by calculating the gradients by hand and implementing them manually into the network.

Rating

6

Confidence

2

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors addressed the limitations, especially the lack of theoretical justification, nicely.

Summary

In their paper ‘Counter-Current Learning: A Biologically Plausible Dual Network Approach for Deep Learning’ the authors introduce their new ‘counter-current learning (CCL) framework’ which they use to train neural networks of multiple network classes (MLPs, CNNs, Autoencoders) using a learning mechanism which solely relies on local calculation of learning signals, without the need to fully propagate gradients through the network. Through this, authors establish a link the brain’s counter-current exchange mechanism. The authors also show an analysis how feature representations are shaped over the course of learning in their learning framework. Whereas many biologically-plausible learning mechanisms have been proposed throughout the last couple of year, the author’s work particularly stands out due to its generality / easy of being applied to various different network classes used in machine learning.

Strengths

As mentioned in the summary, I think many biologically-plausible learning mechanisms have been proposed throughout the last years and the author’s framework stands out to me due to it apparently generality as it seems to easily be applied to network classes which are, from my point of view, not commonly considered in bio-inspired learning mechanism papers, like autoencoders. This generality could allow for the authors framework to have an impact beyond theoretical neuroscience but also influence the more engineering focused side (e.g. neuromorphic computing). The new learning framework is supported by some additional empirical investigations, giving an intuitive understanding of the learning dynamics within the network.

Weaknesses

While authors mention a potential biological link, not much space is given to discussing specific biological implementations and authors do not seem to make specific predictions which would allow for empirical validation of their proposed mechanism in biology, to validate whether the mechanism is actually utilised in the brain. As such, I see this paper as being more focused on the engineering / computing solutions side, where it does a good job, mentioned above. Given this focus on the computing side, I would have hoped that authors would also consider training a model using a current SOTA architecture (i.e. Transformers or State Space Models) using their technique, to show whether their algorithm can also handle these cases. It would be important to discuss how the proposed work relates to a recent line of work in which two streams of information are assumed to exist in the brain: burstprop (Payeur et al. Nature Neuroscience 2021) and burstCCN (Greedy et al. NeurIPS 2022). Moreover, this work seems to have links with Sacramento et al. 2018 NeurIPS, in that both require specialised feedback neurons that encode the error signals. Both Sacramento et al. and Greedy et al. use an interneuron that learns to match the feedback, which conceptually seems to match the idea proposed by the authors. Perhaps even more relevant is the predictive coding of backprop by the group of Rafal Bogacz, which requires separate error neurons and both the forward and backward phases to align with each other. These should be discussed, and it may be worth making a more explicit link in future work.

Questions

If the authors see their work primarily focused on an engineering solution, I am wondering whether their technique generalises to Transformers and / or State Space Models (like Mamba or Griffin). Otherwise, I wonder whether authors could make specific predictions based on the learning dynamics observed in their model which would allow the community to test for its plausibility using neural data. How does your work relate to the bioplausible works referred to above?

Rating

6

Confidence

3

Soundness

2

Presentation

3

Contribution

2

Limitations

I think authors address limitations on the machine learning / engineering side well but could potentially expand their discussion of limitations with regards to biological evidence for / against their proposed learning framework.

Summary

The article proposes a learning framework that addresses three major critiques of the backpropagation algorithm regarding its biological plausibility: (i) the weight transport problem, i.e., the error feedback weights being the transposes of the feedforward weights, (ii) the nonlocal update problem, i.e., the local weights being dependent on the global error signal, and (iii) the backward locking problem, i.e., the need to freeze layer activations until error feedback information becomes available. The basic idea of the proposed framework is to construct a dual network that operates in the reverse direction to the original inference network. This dual-network has transpose dimensional weights (but not necessarily the transposes of the forward weights), with some similarity to the case of backpropagation. However, this dual network propagates the target value backward. Each layer of the dual network is compared to the corresponding layer (of the same dimension) in the original inference network. The loss is defined as the mean square error between the activations of the corresponding layers. The numerical results demonstrate that the test performance in classification task are on par with backpropagation and the state-of-the-art models.

Strengths

- Novelty: The article offers a novel approach for the biologically plausible learning framework. The use of a dual network to propagate the target value backward and using the mean square error between the corresponding representations of the network as the loss function appears to be a new idea. - Numerical Experiments: The article presents several experiments demonstrating the performance of the proposed approach in comparison to existing methods. The performance appears to be close to that of backpropagation (and even significantly better in the case of CIFAR-10).

Weaknesses

- Analytical Support/Discussion: There is no analytical justification provided for the proposed approach, making it unclear why it should work. This is the main weakness of the article. At the very least, there could be more discussion aimed at providing intuition about the underlying mechanism of the proposed approach that would enable effective training of neural networks. The authors cite "counter-current" mechanisms observed in nature as their primary motivation. - Presentation: The presentation is generally smooth and clear. However, it can be further improved as addressed by the questions below. - Reporting of Numerical Experiments: The results reported for the numerical experiments are confusing, especially as they conflict with those of Shibuya et al. [2023], whose experimental setup is adopted in the current article. - Scalability: It is uncertain whether this idea would work for large-scale inputs/models (e.g., ImageNet 1K/ResNets).

Questions

- How does the proposed approach train the forward neural network to minimize the mean square error (MSE) at the final layer? - It would be useful to report the alignment (angle) between the forward and backward network weights, to check if the weight symmetry problem still exists. - The sentence "Due to random weight initialization, the information content ... decreases ..." in the caption of Figure 1(a) is imprecise. How do you measure information content and relative to what, and why is it decreasing? Furthermore, the next sentence does not really apply to the "before training" (a) part of Figure 1. - Figure 3: Do we see a similar organization of representations in other layers? Since the target label input for each class is fixed, we expect the backward network representations to be a single point for each class in any layer. Do the forward network layer activations appear as clouds around these points? - Section 4.2: MLP Experiments: Why do the models used for comparison employ tanh nonlinearity, while CCL uses ELU? Test results with the same model properties would be more meaningful. - Table 1: Is the DTP in the line below FA correct? As Noakland 16 is cited, is it actually DFA? Shibuya et al. [2023], which the authors claim to base their experiments on, report 52.17% accuracy for DTP/CIFAR10, which is not available or conflicts with Table 1. Similarly, Shibuya et al. [2023] report 51.33% for FA-CIFAR10, but Table 1 in the current article reports FA performance as 45.76%. The FW-DTP performances also do not match. - Figure 4: It would make more sense to compare the conv. kernels of CCL after training with the corresponding kernels of BP, as the initial random kernels shown in this figure are not informative. - Section 4.4: Do we need the CKA-based measure to compare the representations of the forward and backward networks? The proposed approach tries to minimize the mean square error (MSE) between them, so do we expect any linear transformation uncertainty? Isn't it more meaningful to look directly at the angle of alignment or the MSE normalized by the norm-square of the forward representations? - Figure 7 and Section 4.5: The best performance appears to occur toward the top right corner, suggesting that feedforward-feedback learning rates should be similar (symmetric) rather than asymmetric?

Rating

6

Confidence

4

Soundness

2

Presentation

3

Contribution

2

Limitations

The authors sufficiently discuss the limitations of their work at the end of the article. Related points are raised in the weaknesses and questions part.