Simultaneous embedding of multiple attractor manifolds in a recurrent neural network using constrained gradient optimization
The storage of continuous variables in working memory is hypothesized to be sustained in the brain by the dynamics of recurrent neural networks (RNNs) whose steady states form continuous manifolds. In some cases, it is thought that the synaptic connectivity supports multiple attractor manifolds, each mapped to a different context or task. For example, in hippocampal area CA3, positions in distinct environments are represented by distinct sets of population activity patterns, each forming a continuum. It has been argued that the embedding of multiple continuous attractors in a single RNN inevitably causes detrimental interference: quenched noise in the synaptic connectivity disrupts the continuity of each attractor, replacing it by a discrete set of steady states that can be conceptualized as lying on local minima of an abstract energy landscape. Consequently, population activity patterns exhibit systematic drifts towards one of these discrete minima, thereby degrading the stored memory over time. Here we show that it is possible to dramatically attenuate these detrimental interference effects by adjusting the synaptic weights. Synaptic weight adjustments are derived from a loss function that quantifies the roughness of the energy landscape along each of the embedded attractor manifolds. By minimizing this loss function, the stability of states can be dramatically improved, without compromising the capacity.
Paper
Similar papers
Peer review
Summary
The paper investigates the challenge of embedding multiple continuous attractor manifolds within a single RNN, with a focus on hippocampal place cells. The issues arise due to the presence of discrete steady states, visualized as minima on an abstract energy landscape, which disrupt the continuity of network activity patterns. This disruption prompts systematic drift of population activity patterns towards these discrete states, resulting in degraded memory over time. Past studies have considered the stabilizing influence of external stimuli; however, solutions in their absence remain unclear. The authors address this issue by modifying the synaptic weight to flatten the energy landscape, showcasing through simulations how this significantly stabilizes the activity pattern.
Strengths
o The methodology and problem formulation are clearly articulated. o Simulations indicate that weight modification significantly improves the stability of activity patterns. o The authors provide the code in the appendix.
Weaknesses
o While the manuscript is generally well-written, some areas could benefit from improvement. A diagrammatic illustration could help elucidate the issue of discretized states in the context of the energy landscape. Further details of the simulations (e.g., duration and specific inputs to the network) need to be provided. The appendix, which helps clarify the methodology, should be referred to more often in the Results section. o Strengthening the paper could involve additional simulations and/or discussions (see questions below). o There is a minor typo: "emfbedded map" should be "embedded map."
Questions
o How applicable is your approach if connection weights are not symmetrical? o How robust is the pattern stability if the modified weights are slightly perturbed? o Can your approach provide biological insights into the structure of weight modification when new maps are added? o Does the resulting weight (after the modification) violate Dale’s law? o While it's evident that an energy landscape embedded with minima leads to activity patterns settling into one of them, thereby degrading the network’s ability to sustain persistent memory, does a flat energy landscape guarantee pattern stability? o Here are some additional related works that could be included in the discussion [1-2]. o References: [1] Genkin and Engel, Nature Machine Intelligence, 2020: https://www.nature.com/articles/s42256-020-00242-6 [2] Whittington et al., Cell, 2020: https://www.sciencedirect.com/science/article/pii/S009286742031388X
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
3 good
Contribution
4 excellent
Limitations
The authors have acknowledged certain limitations, such as the unclear biological plausibility of their approach.
Summary
The paper studies the storage of multiple continuous attractors in a recurrent neural network. Specifically, the authors tackle the interference between attractors and its effect on activity bump drift. By using a perturbative approach, they compute a correction to the connectivity that reduces the drift dramatically. Continuous attractors (e.g., ring model) are important models in neuroscience, and understanding them is an important task. Multiple attractors are relevant, for example, in the CA3 region of the hippocampus, where remapping of place cells in different environments is common. Nevertheless, a naïve connectivity that is a superposition of several ring-connectivities results in only approximate continuous attractors. The result is a few stable points in each attractor, to which dynamics converge. The authors use the perspective of an energy function (Lyapunov), and examine how the interference renders this function non-flat. They then calculate the perturbation to leading order, and solve for a change in connectivity that will flatten the energy. Furthermore, using gradient descent, they are able to achieve even greater precision.
Strengths
Continuous attractors are a fundamental building block in the study of recurrent neural networks in neuroscience contexts. There are relatively few studies tackling multiple such attractors. The method to reduce interference is novel.
Weaknesses
First, as the authors note, the resulting connectivity is extremely fine-tuned. This is a known problem with continuous attractors that is not addressed here. Second, the problem and solution are highly related to similar problems in discrete attractors. Interference in Hopfield networks was tackled using pseudo inverse rules, either approximated online or as a global formula. There is no discussion of the relation to these works. The SVM approach of Battista and Monasson (Ref 5) is perhaps a similar example in continuous attractors.
Questions
L146 energy IS quadratic L178 “vast majority” the sentence isn’t very clear. Was the intention “close to a fixed point” or something similar? L146 – I didn’t fully understand the argument on why the first order vanishes. Can you say something about higher dimensions? Place cells are relevant in 2D. The results of [5,23] suggest differences when dimension increases.
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
4 excellent
Presentation
4 excellent
Contribution
3 good
Limitations
yes
Summary
A new method is proposed to allow the simultaneous embedding of multiple attractors in an RNN through minimization of the energy function corresponding to the dynamics. Two different methods to achieve this are considered, the first one based on the linearized energy function and the second on constrained optimization. The second method optimizes the weights for a flat energy landscape with the constraint that there is no change in the center of mass of the bumps.
Strengths
The aim to flatten the energy landscape of an attractor network with multiple attractors embedded in it is novel. The main aim and the methods are clearly described. The discussed problem of detrimental inference is definitely very important for theoretical neuroscience.
Weaknesses
There is a lack of assurance that the modified network actually has the equalized energy function maintains the bumps as minima of the energy landscape. The procedure also doesn't necessarily contribute to a flat energy landscape, it just ensures that all the evaluated states have the same energy value. A more thorough way to enforce flattening energy functions for RNNs is described in: Noorman, Marcella, et al. "Accurate angular integration with only a handful of neurons." bioRxiv (2022): 2022-05. The time to implement the algorithm is also really long 72 hours for $L=60$ and does not seems to be practical. The reason to use the bump score is not fully justified.
Questions
174: To analyse the modified network, one can use the linearized network or Lyapunov exponents to identify stability. Try to find the equilibria of the modified network and assess the stability of these points. Do the equilibrium states have a Lyapunov exponent close to 0? (This is expected from the ring attractor structure.) CANNs can implement path integration. Is it possible to do that reliably in the proposed network for the different embedded maps? However, the proposed idealized bump method to do this does not guarantee that the acquired energy landscape is truly flat. Can you plot the energy landscape for values in between bump positions? SI 111: Do you mean that the function $f$ is defined \emph{in terms of} the weighted averaged position of the rates? The definition that follows is different. 132-133: Could you further explain how these steps follow from each other? The gradient of $I^{k,l}$ in S24 is not dependent on the constraint, why couldn't the derivative of it with respect to $M_{ij}$ change in another direction? 190: Could you further assess the stability of the network by introducing small displacements in the activity? What are the Lyapunov exponents of the network? Do they correspond to a continuous attractor? 194: Could a different definition of the location lead to a different conclusion from the analysis? The actual attractor states are not the idealized attractors any more necessarily. Therefore, the mean square change defined like this might be biased.
Rating
5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
3 good
Presentation
2 fair
Contribution
2 fair
Limitations
They are adequately discussed.
Summary
This work tackles the problem of interference between continuous attractors when they are held in a single RNN. The authors adopted a Lyapunov function as a depiction of energy of the network and tried to flatten the energy landscape of attractors by adding a modulation term to the original connection matrix. The modulation term was derived by two methods respectively: a first-order approximation of the energy function and a constrained iterative gradient-based method. They showed that by constraining bumps at their initial position, the gradient descent achieved a better result.
Strengths
The work achieved the goal of encoding multiple continuous attractors into a single recurrent network.
Weaknesses
1. The recent experimental data actually showed that in the remapping of cognitive maps in hippocampus, place cells encoding different maps actually have little overlap, i.e., the hippocampus recruits different groups of neurons to form different continuous attractors. In other words, the interference between multiple continuous attractors is only a mathematical problem, not a biological problem. This limits the contribution of this study to neuroscience. 2. There are several issues in this study whose biological plausibility are not justified, the energy function (note the real neuronal connections are not symmetry), the modification of synapse strengths based on the attractors the network has stored, the gradient-based learning method. Overall, the insight of this study to neuroscience is rather limited. 3. Some important references are missed, such as the work of Misha Tsodyks et al. on stroring multiple continuous attractors (PLoS Computational Biology?)
Questions
1. Since the learning method is used to determine M, why the authors use W=\sum_i J_i+M and learn the modification term, why not learn W from the first space?
Rating
4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
3 good
Presentation
3 good
Contribution
2 fair
Limitations
The limitations are discussed.
Summary
The authors present a new technique for embedding multiple attractor manifolds into RNNs. To do so, they first randomly choose a number of attractor manifolds, embed these into an RNN, and then make weight adjustments to smooth out the interference created by multiple manifolds. The authors propose two strategies for weight adjustment, which consider first- and then second-order interference effects. These adjustments are shown to iteratively improve several intuitive metrics.
Strengths
This paper addresses and interesting and well-defined problem. This is not my area of expertise, but the results seem quite general to understanding RNN function, and thus significant. The paper is well written and the results clearly presented. The stability metrics are intuitive. The presented solutions are strikingly effective!
Weaknesses
While the results are very strong, the paper only explores a single task (embedding 1D ring attractors). Do the results hold when moving to, say, two dimensions? How does the number of neurons in the network interact with task complexity?
Questions
the phrase "quenched noise" is used a lot - can this be defined more clearly upon introduction? My idea of "quenched" is equivalent to "reduced", which does not seem to align with how this phrase is being used. L15: typo: adjustment -> adjustments L151: typo: "emfbedded" Fig 1b top: only 9 red lines - need to expand axis limits? Fig 2b top: it is interesting that modification - while flattening the energy landscape - also raises the overall energy of the system. It would be instructive if the authors could offer some intuition for why this is the case. Also, in Fig 4a it appears that the energy after second-order modification is actually less than the single map case - will this always be the case? How should I think about this result? Figs 3-5: what are error bars? The captions say mean+/-SEM, but what are these statistics computed over? Are there multiple network instantiations for each value of L?
Rating
8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
4 excellent
Presentation
4 excellent
Contribution
4 excellent
Limitations
One limitation that is already pointed out by the authors is that their approach is not a biologically plausible learning algorithm; however, I agree that this proof-of-principle is a solid first step and that investigating other mechanisms for weight updates is an interesting direction for future work.
Thank you for the thorough response. Regarding 1D vs 2D environements, this is likely a common question that will come up among readers. I'd suggest at least adding something to this effect, perhaps in the Discussion? > One notable difference between 1D and 2D environments is that the number of neurons required to achieve a good approximation to a continuous attractor, even for a single map, scales in proportion to the area in 2D, as opposed to length in 1D. But for a given number of neurons, there is no substantial difference between the two cases in terms of the complexity of the problem: the number of equations scales as N*L, and the number of parameters (synaptic weights) scales as N^2. Since the random permutations are completely unrelated to the spatial organization of the firing fields, quenched (frozen) noise is expected to behave similarly in the two cases. While I don't think it's strictly necessary to include a 2D example in the manuscript, I think the impact (and at least perceived generality) of this work would increase with said example (even a small one, even in the supplementary). My other concerns/confusions have been adequately addressed.
Thanks for the suggestion, we will mention and address this (2D) in the Discussion of the final version.
Thanks for the authors insightful response. Most of my concerns are resolved. Regarding to the remapping of place cells, the authors could see this work (https://www.pnas.org/doi/10.1073/pnas.1421056111), whose results exhibited that the overlap is minimal between the recruited place cells in different environments.
Thanks for pointing to the manuscript of Alme et al (PNAS, 2014). This work is helpful because it quantifies the overlap more systematically than in previous studies. The recordings made by Alme et al were carried out mostly in novel environments and only for 15 minutes, and therefore it is possible that spatial maps were not yet fully established in the CA3 network (see also Leutgeb et al, 2004). Nevertheless, the recordings are informative on a qualitative level, and the conclusion is that overlap is small, but not minimal. In the context of our work the overlap observed by Alme et al is extensive and consequential, as we explain below. The interpretation reached by Alme et al is that most CA3 place cells are active in ~7% of the environments and a smaller subset of cells, estimated in the paper to be 10%-20% of the total population, are active in a much larger fraction of the environments. Furthermore, the picture emerging from the paper is not of a strict division of cells into clusters, each representing a distinct map. The results are consistent with a picture in which the participation of each cell in any given environment is determined randomly and independently of the other cells. In terms of the whole population, the above estimates suggest that the participation ratio (defined as the fraction of cells that are active in any given map) is somewhere between 10% and 20%. A similar conclusion can be reached by asking what is the probability that a cell will be active in a map B, given that it is active in map A (and is therefore already classified as a place cell). This information can be extracted from the histogram shown in Fig. 4A: the cells that had a field in one map or more participated, on average, in the representation of 2.75 rooms out of 11. Excluding the environment used to classify the cell as a place cell, the participation ratio is approximately no less than 1.75/10 = 0.175 ( (2.75-1)/(11-1)). With a participation ratio of ~0.15, neurons expressing a bump state in one environment are expected to receive numerous synaptic inputs resulting from weights associated with any other map. These will add up in proportion to the number of embedded spatial maps. Furthermore, many studies indicate that the hippocampus encodes memories other than spatial maps, which are expected to contribute as well to the frozen noise. Overall, the effect of frozen noise associated with the embedding of multiple memories will depend on p, on L (the number of spatial maps, and other associated memories) and on N, the number of neurons participating in each map. Importantly, the flatness of each attractor state is inevitably compromised in the naive embedding scheme. The magnitude of the effect must increase in proportion to the number of embedded memories, and therefore it must become large for sufficiently large L. For computational simplicity we implemented in our simulations a network architecture in which all cells participate in all maps. However, it would be straightforward to adapt the architecture to one with a participation ratio smaller than 1. We agree that it will be interesting in a follow-up study to examine (numerically and using analytical tools) the interplay between the participation ratio p, the number of embedded maps L, and the number of neurons participating in each map N. Note that small p implies low interference, which is beneficial since it reduces the frozen noise, but when keeping the total number of CA3 neurons fixed, it also implies a reduction in the number of neurons participating in each map, and in each bump state. This is expected to reduce the resilience of each attractor to the frozen noise. We thank the reviewer for raising these questions. We will add a paragraph in the Discussion of the final version, in which we will discuss the fact that in reality, neurons in CA3 participate only in a subset of maps, and how this could be addressed in future studies.
Response acknowledged
Thank you for your thorough response and additions to the paper. I am going to increase my score.
post rebuttal
Thank you for the replies and clarifications. Regarding the approach of ensuring a number of patterns are fixed points, and the relation to continuous attractors, this reference might also be relevant: Darshan, R., & Rivkind, A. (2022). Learning to represent continuous variables in heterogeneous neural networks. Cell Reports, 39(1). I'm maintaining my score.
Thanks for the suggestion, we agree that this reference is relevant in this context and will include it in the final version.
I thank the authors for their replies and clarifications. About the flatness of the energy landscape When do you consider the number of neurons to be small? I agree that truly continuous attractors are only achieved when the number of neurons is infinite, but that would imply that anything smaller than that is a "small" number of neurons? Is there a different sense in which you understand "small" for the "practical significance" you mention? There seems to be a misunderstanding about my reference to the Lyapunov exponent (https://en.wikipedia.org/wiki/Lyapunov_exponent). I understand Fig 5 to contain information about the Lyapunov energy, however, I think it would benefit the analysis and would make the claims stronger about stability. To me it is unclear how the measure of stability in the paper relates to a resolution that is below the spatial resolution of one neuron (which should be considered for a continuous attractor). I maintain my score.
>About the flatness of the energy landscape When do you consider the number of neurons to be small? I agree that truly continuous attractors are only achieved when the number of neurons is infinite, but that would imply that anything smaller than that is a "small" number of neurons? Is there a different sense in which you understand "small" for the "practical significance" you mention? There are several ways in which we understand "small" from a practical standpoint. The first is simple: if there are, say, a thousand neurons participating in a ring attractor with a single map, the graininess of the energy landscape is 2pi/1000. In other words, a stable representation can be maintained at the resolution of a full circle divided by the number of neurons. Whether or not this is small in a practical sense depends on the context, but it is no coincidence that the question addressed in the work of Noorman et al was brought up only recently, with the discovery of a ring attractor with a few dozens of neurons. In CA3, one can estimate that tens of thousands of neurons participate in the representation of each square meter map. Note, in addition, that the granularity arising from the finite number of neurons is small, within the parameters that we work with, compared to the tuning curve of each neuron. Second, the drifts that we address in our work, which arise from the embedding of multiple maps, are much larger than the single neuron resolution - see Figure 3d. The single neuron resolution seems to us like a natural scale for our attempts to flatten the energy landscape, since this is the graininess which is expected without further manipulations even if only a single map is embedded in the connectivity. In principle, however, we could have attempted to equalize the energy over a set of positions which is more dense than the single neuron resolution. Our methodology can be applied to achieve this goal even in the single map case - in similarity to the results of Noorman et al, but using an approach which is in some ways more general. We were able to do so for various forms of the single-map connectivity, and were even able to obtain an exquisitely stable representation of positions over a continuum of angles in a network consisting of only three neurons. We did not include these results in our submission because the focus of the present work is on the consequences arising from the embedding of multiple maps. Third, in a realistic neural network one expects to have two types of noise: frozen noise in the connectivity, which in our context arises from the embedding of multiple maps, and dynamic noise that arises from the fact that neural activity is dynamically stochastic. Dynamic noise causes diffusive random motion in the position of the bump, which accumulates over time. The characteristic magnitude of the diffusive motion over a short time interval Dt scales as the square root of Dt/N, where N is number of neurons, whereas the graininess arising from the discrete number of neurons scales as one over N. In addition, the energy barriers reduce with N.. Consequently, random diffusive motion dominates the motion over short time scales, and the systematic drifts associated with the discrete number of neurons are completely washed out by the random diffusive motion when N is sufficiently large. With biophysically reasonable parameters, and when N is in the order of several hundred or more, the statistics of random motion in a single ring attractor is practically indistinguishable from the statistics predicted analytically using the continuum limit, while completely ignoring the graininess that arises from the finite number of neurons (see, e.g., https://doi.org/10.1073/pnas.1117386109). >There seems to be a misunderstanding about my reference to the Lyapunov exponent (https://en.wikipedia.org/wiki/Lyapunov_exponent). I understand Fig 5 to contain information about the Lyapunov energy, however, I think it would benefit the analysis and would make the claims stronger about stability. To me it is unclear how the measure of stability in the paper relates to a resolution that is below the spatial resolution of one neuron (which should be considered for a continuous attractor). Lyapunov exponents are primarily used to identify chaotic dynamics. In our case, there is no question that the system has stable attractors, and it cannot have a positive Lyapunov exponent. You are correct, that a system with a semi-stable continuous attractor should have a vanishing Lyapunov exponent. What we pointed out in our previous response is that in a system with stable attractors, a somewhat simpler signature for the continuity is that the Jacobian has a vanishing eigenvalue. This has indeed been used previously in the literature on attractor neural networks as a way to quantify the continuity of the attractor, and we could add such an analysis to the final version.
Decision
Accept (poster)