Adaptive whitening with fast gain modulation and slow synaptic plasticity

Summary

This paper proposes a normative principle for the symmetric whitening problem, i.e., a batch optimization problem which is provable to attain symmetric whitening is used to derive an online adaptive algorithm. The proposed framework unifies the previous works, and the resulting algorithm maps into single layer network with interneurons and local learning rules. The mathematical findings of this normative approach is also supported with the evidences from neuroscience. The proposed approached is tested with synthetic data and natural images to demonstrate the performance. Potential improvements for more biologically realistic method is also discussed.

Strengths

The paper provides a neural network solution to the whitening problem which satisfies two important constraints for biological plausibility: i) the network operates in an online manner, and ii) the weight and gain updates are local. The proposed online algorithm is illustrated to obtain whitening with both synthetic and natural datasets.

Weaknesses

The writing is lacking in some parts, and some important details are missing in the paper. More numerical experiments should be presented to assess the introduced framework. For exact details on the weaknesses, please see my comments below and questions section. * Line 83, "... where $f_c(\mathbf{M}_c) = 2 \mathbf{M}_c$" should be changed to "... where $f_c(\mathbf{M}_c) = \text{Tr}(2 \mathbf{M}_c)$" since $f_c$ is a scalar function of matrices. Moreover, even though it is a simple proof, it would be valuable how this unique minimum is achieved. * The intermediate step (stated between line 150 and line 153) to obtain Equation (4) should be written explicitly in the paper (can be in the appendix). * The decoupling the feedforward and feedback weights (in Appendix D.2) is very similar to the idea presented in [i] since the assumption $g_i + m_i \in (0, 2 \eta_w^{-1})$ corresponds to weight decaying. The paper should cite this reference. [i]J.F. Kolen and J.B. Pollack. Backpropagation without weight transport. In Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN’94), volume 3, pages 1375–1380 vol.3, 1994. doi: 10.1109/ICNN.1994.374486. * For the experiment with synthetic data, the experiment only considers the whitening matrix of the form $\mathbf{M}_c = \mathbf{I}_N + \mathbf{V} \mathbf{\Lambda}(c)\mathbf{V}^T$. An experiment with more general form of whitening matrix is required to assess the performance of the proposed algorithm. * Hyperparameters (i.e., $\eta_w, \eta_g, \eta_r$) are not provided in the paper (except that the authors provide $\eta_w$ for Algorithm 2 in Appendix C). It would be beneficial for the paper to include a table that provides the hyperparameters used in each experiment * The differences and novelties compared to previous works by Pehlevan and Chklovskii [11] and Duong et al. [18] are discussed quite well, but the comparison in the numerical experiments is missing. * A single Python script is shared for the proposed method, although it does not include the experiment code. The authors mentioned in the paper that the full code would be shared upon publication. However, the current code does not provide a comprehensive understanding of the conducted experiments.

Questions

* The numerical experiments presented in Appendix C uses Algorithm 2, but the biological plausibility of it is questionable due to matrix inversion and batch learning. Did you experiment with this setup using Algorithm 1? * The authors propose Algorithm 3 as more biologically plausible algorithm compared to the Algorithm 1, by complying to the Dale's law and decoupling the feedforward and feedback weights for asymmetry (weight transport problem). However, no numerical experiment has been demonstrated for this proposal. Did you test this algorithm? * The value of $\alpha$ seems to be also an hyperparameter for the proposed framework. Did you analyze the effect of this choice?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

2 fair

Limitations

I think that the limitations are adequately discussed in the paper.

Summary

This paper gives an algorithm for learning weights of a neural network over long time scales, which allow interneurons to decorrelate the responses of excitatory neurons by modulating their gains over short time scales. On both synthetic and natural image datasets, this algorithm is shown to be effective and to generalize to new contexts.

Strengths

* This paper makes progress on a fundamental question in neuroscience: How a population of neurons can coordinate their responses to stimuli to encode it. This work should also be of interest in machine learning, as it has obvious applications to transfer learning. * Overall, the paper is clear and easy to understand. * The learning algorithm is online and local. * The technical evaluations are fairly thorough and support the paper's claims.

Weaknesses

Some weaknesses are inherited from the Duong et al. model which this paper is building on, specifically related to its biological plausibility: * All neurons are linear. * Dale's law is not observed. * Forward and backward weights are mirrored. * In general, more interneurons than excitatory neurons are required (in the worst case, as many as $\Omega(n^2)$ interneurons for $n$ excitatory neurons), which does not match the ratios observed in the brain. Experimentally, the algorithm fails gracefully when fewer interneurons than required are available. A more realistic version of the model which addresses the first two issues is provided in the appendix. However, given that the scope of this work is primarily modeling a phenomenon in neuroscience, I think the paper would be stronger if this biologically plausible version were featured and evaluated more prominently.

Questions

* Line 83: Missing a trace on the RHS of the equation? * I would appreciate a more in-depth evaluation, perhaps on a synthetic dataset, of what happens when there are fewer interneurons than required. * Did you consider adding an activation function, at least to the interneurons? I could imagine that there is a similar learning algorithm which allows for fewer nonlinear interneurons.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

2 fair

Limitations

Yes.

Summary

The authors produce a mechanistic model that combines synaptic plasticity and gain modulation to adaptively whiten responses. This model is constructed from an objective for learning a whitening transformation and then considering matrix factorization. Simple factorization introduces interneurons and optimization via gradient descent can be mapped to synaptic plasticity which adapts the weights to a particular context. Another decomposition that enforces a particular diagonalization leads whitening via gain modulation when the weights of the interneurons are fixed. The contribution of this paper is to combine these two mechanisms into a single cost function and network instantiation in which the gain optimization produces a whitening for a given context, and the synaptic plasticity adapts to properties useful over multiple contexts. The describe an implementation of this optimization in a recurrent neural network and consider different time scales, in which gain adaptation happens quickly within a context and synaptic plasticity happens over a longer timescale. The authors test this framework on simulated data that matches the decomposition assumptions and show that the method is successful. They further test it on natural images and show that the learned connectivity successfully adapts to new contexts.

Strengths

This is a very nice paper that connects the computational problem of whitening with multiple neural mechanisms into a single normative framework. While it still has issues with full biological plausibility (as noted by the authors) this is an interesting step in thinking about this computation and the roles of these neural mechanisms. The presentation is very clear (Figure 2 is particularly nice).

Weaknesses

There are no significant weaknesses in this paper. To the authors’ credit, every question I had that I thought was in the category of “necessary for publication” they addressed. I’ll leave everything else to “Questions”, below. The biggest weakness I see regards the generality of the matrix decomposition. The extent to which the authors test or address this appears on par with other approaches in the field, though. I have some questions below on this topic.

Questions

Can you make any statements about how general the matrix decomposition in Section 3.3 is? It seems very specific. You demonstrate numerically that this decomposition works for your experiment. When do you expect it to fail? Can you characterize or quantify the differences in context necessary before this decomposition no longer works? Related to this, it looks like you’re using images from the van Hateren database. Can you try the held out image experiments with images from very different contexts and origins, since those images all have clear similarities in large scale properties? This might help find the limits of this algorithm/decomposition. I was going to ask about breaking the symmetry on the weights, which you answered in an appendix. They will asymptotically align, but can you say something about this timescale? Is there a restriction on when the alignment has to happen with respect to the gain modulation and synaptic plasticity that is meaningful or prohibitive? You provide the error as a function of number of interneurons for your natural image experiment. Can you compute some normalization so that there is an idea of what errors are meaningful (maybe the Frobenius norm of C)? Your interneurons are not necessarily the “interneurons” people talk about in real circuits, but it makes one imagine nonetheless that you will have far fewer interneurons than primary neurons. Is this a functional problem given the amount of error you see? Maybe it’s “good enough” even with few interneurons?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

4 excellent

Presentation

4 excellent

Contribution

3 good

Limitations

The authors do not discuss societal impact.

Summary

This paper proposes a neural circuit model that combines fast gain modulation and slow synaptic plasticity to adaptively white sensory inputs. It appears that this paper is a combination of the studies of ref. 11 and 18.

Strengths

### Originality The strength of this paper is that it combines the fast gain modulation and slow synaptic plasticity in a whitening neural circuit, and addresses the shortcomings of earlier models with either gain modulation or synaptic plasticity. This is the most conceptual and technical advance of this study. ### Clarity Overall, the paper is well written. But see my comments and questions on some parts I am unclear about.

Weaknesses

I don't have major concerns about this study from the math point of view. Nonetheless, I have a significant concern from the neuroscience point of view. The derived updating rule of gain (Eq. after line 174) depends on the current weight. The neural "gain" is typically adjusted by inhibitory interneurons, and thus I am concerned about how interneurons "sense" the synaptic weights. If there are neurobiological studies supporting this updating rule, please cite the reference in the paper, otherwise, it is better to discuss it at the end.

Questions

- Line 91: ref. 11 requires $K \geq N$, i.e., the number of I neurons is larger than the E neurons. Does this requirement also need in this paper? Is there a way to make the number of I neurons less than the E neurons? This is also the case in the cortex. - It seems that the $f_c(M)$ function was used before but the writing still looks abrupt to me about why to propose an objective function with such a form. - The paragraph starting at line 90. It is better to first explain the decomposition $M_c = W_c W_c^T$ regards to the weights between E and I neurons. Otherwise, people can easily regard $W_c$ as the feedforward weights.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

I realize the framework assumes the synaptic weights between excitatory (E) and inhibitory (I) neurons are symmetric with each other by just differing a sign. Is there a way to break this symmetry? Will the symmetry breaking of E-I weights facilitate optimization?

Response to the Rebuttal by Authors

I would like thank to the authors for their thoughtful responses. I genuinely appreciate the central contributions made by this work. My intention in offering my comments was to provide constructive insights on aspects that, in my opinion, could be beneficial for further refinement. In this context, I find that this rebuttal and global rebuttal address the concerns and queries I had raised. I think that the promised revisions and provided clarifications have already enhanced or will further enhance the current manuscript. I would like to also thank for the supplementary experiments presented in the global PDF, which are valuable additions to the work. As a result of these considerations, I am inclined to adjust my rating from 5 to 6.