Beyond Geometry: Comparing the Temporal Structure of Computation in Neural Circuits with Dynamical Similarity Analysis
How can we tell whether two neural networks utilize the same internal processes for a particular computation? This question is pertinent for multiple subfields of neuroscience and machine learning, including neuroAI, mechanistic interpretability, and brain-machine interfaces. Standard approaches for comparing neural networks focus on the spatial geometry of latent states. Yet in recurrent networks, computations are implemented at the level of dynamics, and two networks performing the same computation with equivalent dynamics need not exhibit the same geometry. To bridge this gap, we introduce a novel similarity metric that compares two systems at the level of their dynamics, called Dynamical Similarity Analysis (DSA). Our method incorporates two components: Using recent advances in data-driven dynamical systems theory, we learn a high-dimensional linear system that accurately captures core features of the original nonlinear dynamics. Next, we compare different systems passed through this embedding using a novel extension of Procrustes Analysis that accounts for how vector fields change under orthogonal transformation. In four case studies, we demonstrate that our method disentangles conjugate and non-conjugate recurrent neural networks (RNNs), while geometric methods fall short. We additionally show that our method can distinguish learning rules in an unsupervised manner. Our method opens the door to comparative analyses of the essential temporal structure of computation in neural circuits.
Paper
Similar papers
Peer review
Summary
In this paper the authors propose a similarity metric to compare dynamical systems – in particular neural networks. They use a set of recent methods to transform a general dynamical system into a basis in which the dynamics is approximately linear. They compare two dynamical systems by comparing their linear dynamics in this transformed space by seeing how well one system can be turned into the other using only an orthogonal transformation. They test their metric on a variety of settings.
Strengths
I think the idea is a good one. I think it has been well executed. The experiments were well chosen. The writing was generally pretty good.
Weaknesses
I don’t understand why section 2.3 was separate from section 3. It feels like section 2.3 played two roles – advertising the set of experiments that were going to test this new metric in interesting ways, and describing the experimental setup. The first of these is done in the first paragraph, and the second should, in my opinion, be moved into section 3. The separation between setup and results meant I had to keep flipping back, and you repeated yourself for no reason. It would make much more sense to me if each experiment was introduced and its results were discussed together. I thought some of the experimental details were also unclear, and generally you rely too much on the reader recalling in detail many of the papers you cite. In particular: 1. I did not know what the 3-bit flip flop task was before reading this, and had to go find another paper that describes it. It seems crazy not to put a short description in the paper, even if it is in the appendix if page limits are the issue. 2. When trying to understand the ring attractor I eventually decided that r and s in equation 5 were phi(g) and g in equation 25. But then why change notation? And if I was right could this be made clear? If not, it was obviously not clear enough for this dumbo to understand what was going on. 3. Further, in appendix 9 you say “in our deformation analysis we only changed phi after simulation, which does not affect the ring topology but modifies the geometry” I had no idea what this meant. In my head simulation is running the network dynamics, so you’re saying you only changed the non-linearity after you’d simulated, in which case it would have… zero effect. I must be misunderstanding? The claim that DSA measures topological structure of the dynamics would be fairly remarkable and does seem to be backed up by the experiments. However it doesn’t seem at all proven – as you yourself point out in the discussion. In fact, what exactly is being compared through the dynamics of the eigenspace of these lagged timeseries is a little opaque to me – and I might wager to you too. Given this lack of certainty, I have two specific places I think the writing could use a fudge word, like in the discussion where you say “DSA _may_ be capturing similarity at the level of topology”. Namely: 1. Line 251: We demonstrate DSA compares two dynamical systems at the level of their topology. 2. Line 266: This suggests that DSA identifies similarity at the level of the topology… Could you just write '_may identify similarity_', or '_is consistent with comparing at the level of topology_', because the pure claim “We demonstrate DSA compares two dynamical systems at the level of their topology” does not seem well founded.
Questions
- It seems like the scale of the MDS dimensions is being argued to be meaningful in your plots. Beyond some vague intuition that bigger means more variance, are the dimensions meaningful? What do their scales correspond to? - How well do the linearised dynamics capture the underlying network dynamics? In my limited knowledge of the HAVOK methodology (gathered from Steve Brunton’s excellent youtube videos) the system cannot usually be modelled as a linear system without including a forcing term. Do you find the same thing? If so do you not think it could be that key aspects of the dynamics are entering through the dynamics of the forcing term, slightly weakening the power of your method. Regardless, I think it would be good to persuade people that these approximate dynamics you are comparing are actually good approximations. - I did not buy one claim, could you elaborate: “By linearity of PCA, the dynamics of the original system within the PC subspace are preserved” - Line 224. Take the limit, say the dynamics of the system is just circling round an ellipse and you perform PCA to reduce that to 1D, the major axis of the ellipse. Sure, the dynamics on that 1D are the same as the original system but surely you lose something in this projection? And wouldn't that potentially be true of your original system too? - You claim that all networks that solve the 3-bit Flip-Flop task have the same 8 fixed point structure, line 219, do you have any evidence for this? Or is this from Maheswaranathan et al. 2019? If from Maheswaranathan, make it clear, cite them straight after you make the claim! - Why did you not choose figure 3 to also back up your claim that DSA might be capturing similarity at a topological level? (line 317) - Related to the topological claim, my intuition must be off. Humour me for a moment and ignore the time lagging: if you had activity rotating on some ring, then morphed the behaviour to become ellipsoidal, the topology would be the same, but the rotation matrices would be different. Further, the two matrices could not be mapped into one another by an orthogonal transformation. As such, in my intuition your method would (potentially very reasonably) say these were different dynamics even if the topology was the same. Given this, why do you think your method is going to care only about topology? Is it the time-lagging doing some magic?
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
4 excellent
Presentation
3 good
Contribution
3 good
Limitations
The authors include a section of limitations that sound reasonable.
Summary
Understanding how different neural populations process a particular computation is critical for the study of brain computation, the development of brain-inspired technologies like brain-computer interfaces (BCI), and AI applications. Existing methods compare the underlying representations based on the spatial geometry of the latents. However, they fail to capture the dynamics, which are believed to drive the specific computation. To overcome this limitation, this work introduces a new analysis method that first extracts the dynamical structure of the neural network and then aligns these representations for comparison. They test the model when applied to several case studies training RNNs across different tasks and architectures. The proposed analysis can capture similarities between the computations across different networks while other models fail.
Strengths
The paper is clearly presented and technically sound. The method was tested and shown to work well when applied to artificial neural networks trained to solve different tasks and architectures, resulting in different dynamics and geometries. The analysis expands on two existing methods and combines them to understand the comparison between neural dynamics, which support the computation. Understanding similarities and differences between neural networks solving similar tasks is critical for the fields of neuroscience and AI. In this work, they showed how this analysis can identify overlap between computation solutions when other alternative methods fail. Moreover, the authors suggest that it could be used to understand biological network dynamics.
Weaknesses
While the authors provide compelling results supporting the relevance of this metric when applied to RNNs, they have not applied it to neural data. Since this project is largely motivated by the potential study of biological neural circuits and the application to BCIs, applying the analysis to biological data is critical to fully grasp the scope of applications of the proposed method and therefore its significance. As mentioned in the manuscript, the analysis is "[...] useful for experimental neuroscientists due to its simplicity of implementation and ease of interpretation due to its linearity." However, while its linearity provides the aforementioned benefits, it also imposes some limitations that could be interesting to explore. Lastly, when it comes to the application of alignment for BCI applications, it is important to note that the reported training time is between one hour and one day, making it impossible to use in its current state for practical applications when daily alignment is presumably needed. Along the same line of thought, it would be interesting to explore and note the data demands for the new method to work robustly.
Questions
The case studies using RNNs show the strengths of the proposed analysis in comparison to other methods. However, it is not always the case that the new analysis is superior to the alternatives. It would be interesting to explore cases when it isn't and why that is the case. As described, the modified Procrustes alignment seems to only support pairwise comparisons. If that is the case, are there limitations when comparing multiple networks? Does one have to set a reference network to compare all the others?
Rating
5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
3 good
Contribution
3 good
Limitations
The authors thoroughly address the limitations on hyperparameter training and computational cost.
Summary
The authors developed a new computational tool named Dynamical Similarity Analysis (DSA) to measure the similarity between two systems focusing on the dynamics. They constructed the method by combining and modifying Dynamical Mode Decomposition and Statistical Shape Analysis. Their method successfully identified dynamical similarities between systems with different geometries, and conversely distinguished systems with different dynamics with similar geometries. They also used their method to distinguish learning rules underlying measured neural dynamics.
Strengths
I enjoyed this paper, which I found to be both elegant and useful. This work is a novel combination of previously developed techniques. This work is significant because the standard approach for comparing recurrent neural networks merely focuses on the spatial similarities of latent states, ignoring the importance of the difference between temporal similarities. Their proposal avoids the limitations of traditional tools by accounting for neural dynamics. Overall, the authors have clearly delivered their statement, proposed method, experiment, and results. The method is well-grounded and the ideas behind the proposed technique are well-explained and justified. The writing was clear, and the figures were largely appealing (although some of the labels became small and/or blurry). They analyzed when their method is superior to traditional methods, and used simple tasks to demonstrate this superiority. The distinguishing of learning rules from data was particularly nice.
Weaknesses
The authors could better explain and motivate the conventional spatial Procrustes Analysis, as that would help some readers take the next step for the temporal version. They could clarify the interpretation of their similarity transform CAC^-1, as I felt that the explanation that C^-1 “rotates [the vectors’] positions as well” was not a great explanation. Minor: It’s strange to talk favorably in Figure 2 about DSA giving better accuracy but having lower accuracy scores, when discriminating between identical computations. If pairs of networks are correctly identified as “the same dynamics” then NOT distinguishing the networks should have higher accuracy. You might want “discriminability” on the vertical axis of Figure 2d.
Questions
None. Very nice work.
Rating
8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
4 excellent
Presentation
4 excellent
Contribution
4 excellent
Limitations
The authors adequately addressed limitations.
Summary
The authors propose Dynamical Similarity Analysis (DSA), a method for assessing the dynamical similarity between dynamical systems. The method combines ideas from the data-driven Koopman operator literature and a Procrustes-type distance between linear operators. By focusing on dynamics rather than geometry, the method is able to reveal the similarities between RNNs of different architectures trained to perform the same tasks and distinguish between dynamical systems with similar time-averaged states but different underlying dynamics.
Strengths
1. The method is simple and appealing. Appealing use of time delay embeddings -- transform the temporal task into a geometric task. 2. DSA works very well on most of the experiments: Figures 1, 2, 3, and 5.
Weaknesses
1. Should compare to baseline spectral method, for example PCs of CPSD matrix... 2. Figure 4: DSA is not zero. l262 speculates about numerical approximation error, but this seems to be fairly large. Ideally, the line for DSA would stay very close to zero. If this is not due to numerical approximation error, should this be taken as a failure case of DSA? The text mentions the low variability of the DSA distance across values of beta and C, but it seems like the magnitude is more relevant in this example. 3. The manuscript would improve with better theoretical grounding. For example, something in the vein of what is proposed in the paragraph starting at line 317 would greatly improve the paper.
Questions
1. Figure 1 uses angular distance -- where is this defined? 2. There are several comments made about noise in the manuscript. For example, l58: "DSA factors noise into its estimate of neural dynamics." Also l172. I remain confused about how noise interacts with different aspects of the model. For example, how does Takens' theorem interact with noise? 3. How are lags and rank chosen for each experiment. Are the results robust to these hyperparameters? They seem to vary widely from experiment to experiment. Suggestions/comments: * Define what shape metrics are, or perhaps explicitly define one common one. * Condition is not defined in the setup. * Quantifying dissimilarity: Figure 3: MDS is a good visualization, but there should also be predictions as in Figure 2 or distances as in Figure 1. This should be uniform across figures.
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
Soundness
3 good
Presentation
3 good
Contribution
2 fair
Limitations
* Paragraph starting at l333: I would expect one would need to fit a model that fits task-relevant dynamics. That is, DSA would probably require some modifications before it is applied to neural data.
Reviewer Response
I thank the authors for their thorough response. Some comments: First, regarding the nonlinearity you applied to the ring attractor network, I think I understand now that you ran ring attractor dynamics, got a set of firing rates through time, then mapped those through some nonlinearity to create a morphed version of the same ring as shown in figure 4. If so that now makes sense to me but could definitely be more clearly explained. Further I liked another reviewer's worry that the DSA does not equal 0, and that you have worked out a way of making it equal 0 (though it does bring concerns about how hard the transformation is to optimise). Second, I'm glad to hear you have a proof that your method extracts the topological structure, that's very cool. If it is correct then there's no need for including the fudge words! (Unless I'm misunderstanding what the fudge words are for, surely you want to make the stronger claim if you have the theoretical work to support it?) I did not understand the author's explanation about why the forcing term doesn't matter. I think I get that maybe you only care about the internal dynamics operator (the Koopman operator) and perhaps it is true that with or without forcing you'd expect to extract the same Koopman operator, is that what you are saying? To the statement you make about autonomous nonlinear dyanmics in your response, is that not what you are doing? I accept your argument that if all the activity is in a small set of PCs then the dynamics within that space should be the same as the whole system, and I would indeed be satisfied if you either deleted that claim or qualified it, as you suggest. Finally, this is more for my own curiosity, but could you explain what you mean by 'singular values are factored out' in your last piece of response? My intuition is still off! It seems like the example I presented, the ellipse (and you are right, this is basically figure 4c), was wrong because the time lagging is key, so I tried thinking about the effect of time lagging. What I don't get is let's say I create a load of time-lagged versions of the same co-ordinates and then do eigendecomposition, then I would expect the areas of space in which the activity spends more time to be over-represented, upregulating their weight in the eigendecomposition. I think this would be the furthest points of the ellipse, which are those along which most of the data variance lies anyway, so it seems like lagging only make the situation worse?! Does this argument make sense? Do you know what's going wrong?
Response to Reviewer TLG4's Response
Thank you for your comments, and we are glad to be able to have satisfied some of your concerns. We will certainly implement your feedback in our revisions. Here are the answers to your outstanding questions: > perhaps it is true that with or without forcing you'd expect to extract the same Koopman operator, is that what you are saying? This is exactly correct, the same model fitting algorithm (reduced-rank regression) is used in each scenario. The forcing term is chosen after fitting the HAVOK model, by selecting the last r bases of the regression model. That is, the DMD matrix with $n$ modes and $r$ forcing terms is identified by learning a DMD matrix with $n+r$ modes. This means that fitting a HAVOK model with $n$ modes and no forcing would learn the same DMD matrix. > To the statement you make about autonomous nonlinear dynamics in your response, is that not what you are doing? In our case we are using HAVOK for its statistical properties in identifying the DMD matrix, not for its generative properties. We can illustrate what we mean by generating autonomous nonlinear dynamics with an example: Consider the HAVOK model fit to the Lorenz attractor--in the non-forced DMD setting, running this model autonomously (i.e. evolving the HAVOK model via tail-biting - generating predictions recursively based on prior predictions) cannot regenerate the full chaotic dynamics. This is because it is a linear model. Here we mean it does not regenerate the full dynamics in that the model cannot switch lobes of the attractor. The forcing term can be used to remedy this issue and allow for autonomous chaotic dynamics generated by the tail-biting procedure. > Finally, this is more for my own curiosity, but could you explain what you mean by 'singular values are factored out' in your last piece of response? Of course. Let's consider how this relates to PCA whitening (and we will add this to the supplementary information, as this question is very important and worth spending more time explaining). In PCA whitening, we are able to normalize the covariance matrix of the data: $HH^T = U\Sigma^2U^{-1}$, where $H$ is the (already-centered) data matrix (Hankel Matrix), $U$ is the eigenvector matrix of the covariance matrix, and $\Sigma^2$ are the eigenvalues, which are equivalent to the squared singular values of the data. Then the PCA whitening on our data H is: $H \leftarrow \Sigma^{-1}U^{-1}H$ Now, consider $H = U\Sigma V^T$. PCA whitening mathematically amounts to the following expression: $H \leftarrow \Sigma^{-1}U^{-1}U\Sigma V^T = V^T$ as we cancel terms. Thus PCA whitening is equivalent to simply extracting the right singular vectors from the SVD, which is exactly the transformation of the data on which we fit our DMD matrices! Thus we are able to 'sphere' our data just like in PCA whitening. Here, sphering is what is meant by "factoring out the singular values". That being said, we agree that we need to clarify what we mean by "factoring out the singular values" and will substitute this explanation instead. Thank you for your comment.
i thank the authors for their through and detail answers. I believe that there is a lot of potential for this method. Specially since the computational demand is not that high. Still, in order to demonstrate the true value of this approach it would have to be tested on neural data. I am looking forward to your future work.
**Comparing to a spectral method:** I apologize to the authors for how vague my suggestion was in the original review. My concern was that the proposed method uses time information via DMD while the competing approach (Williams et al. 2022) does not use time information. I recognize that part of the contribution of this work is to extend the general sort of analysis in Williams et al. (2022) to the time domain and agree that this a valuable contribution. However, apart from this aspect of the experiment, the comparisons presented in Figures 2b, 3b, and others is possibly misleading/unfair because the competing method do not have access to any time information. This leads me to ask: could there be a stronger baseline which i) does have access to time information and ii) uses existing approaches? In my opinion, this would provide more relevant context for interpreting the experimental results. Here is one concrete proposal that fits these two criteria: apply the method of Williams et al. (2022) to a spectral representation of the data. In more detail, given a single timeseries $X \in \mathbb{R}^{n \times t}$, estimate the cross power spectral density (for example, using ``scipy.signal.csd`` and taking the absolute values, discarding phase information) to create $\tilde{X} \in \mathbb{R}^{n \times n \times f}$ where $f$ denotes the number of frequency bins. Then simply treat these as features to input to the Procrustes metric (Eq. 4). **Theoretical grounding:** I am glad that the authors have found a more explicit theoretical grounding for their method, which will substantially improve the paper. **Figure 4:** I am glad that the issue in Figure 4 has been fixed. **Noise:** I thank the authors for pointing out Casdagli et al. (1991). Is it fair to summarize the noise situation, meaning nondeterministic dynamics, as follows? i) The theory appealed to in motivating the method (Taken's theorem) was developed for deterministic settings. ii) There have been attempts to systematically study the effects of noise (eg Casdagli et al., 1991) and it has been found that noise somewhat complicates the task of state space reconstruction. iii) Small amounts of noise are helpful in practice for estimating the dynamics matrices and distinguishing different dynamics (as in Figure 3). Is this a fair characterization, I suggest mentioning all three points explicitly in the manuscript. **Matrix similarity:** The orthogonal group of matrices consists of two connected components, but the Cayley transform used (Eq. 5 in the supplementary material) produces only matrices in the special orthogonal group. Due to this, it seems that references to $O(n)$ in the text should be replaced with $SO(n)$. A minor suggestion: it may be helpful for clarity to point out in the text around Eq. 4 that $d=0$ implies the two $A$ matrices are (unitarily) similar. It would also be helpful to mention in the main text that the converse is not true in general, due to the restriction of $C$ to $SO(n)$. I'm curious if the authors tried optimizing the generic matrix similarity metric (Eq. 4 in the supplement) using gradient descent. Does the nonconvexity pose a practical problem? On a related note, it may be safe to assume that both $A$ matrices contain distinct eigenvalues, given that they are estimated using noisy experimental data. If this is the case, it may be possible to define an analogous metric to DSA (call it DSA') by computing a metric on the eigenvalues themselves, for example a Wasserstein distance between the two sets of eigenvalues. This would have the advantage of the following implication: if the eigenvalues of the A matrices are distinct and DSA' is 0, then the A matrices are similar. Note that this is slightly more general than what can be said about the existing DSA. Of course DSA' also depends practically on the numerical stability of the eigenvalue estimation. **One more minor comment:** It seems like DSA is not invariant to changes in timescales. For example, if we have two identical systems with one evolving a few percent faster than the other, the DSA score will be nonzero. This could be worth mentioning as a limitation, given that we may not expect neural dynamics to operate at identical speeds on different trials or across different animals. Thank you to the authors for your detailed response. Most of my concerns have been adequately addressed, and so I will raise my score. However, I still think the manuscript would be substantially improved with some comparison to a spectral method like the one described above. Apologies for my slow response.
We thank the reviewer for their extensive comments, and greatly appreciate their raising of the score. We are glad to have addressed your concerns, and believe that the points you have raised in this response are quite valuable. **Comparing to a spectral method**: We believe that we understand the proposed spectral method, and agree that it serves as another relevant baseline to cover both the purely spatial and the purely temporal view of dynamics. **Noise**: Yes, your points are well-described and we will add them to the manuscript. **Matrix similarity**: You are correct that the Cayley Map restricts to $SO(n)$, and we will change the text accordingly. Likewise, you are correct that the metrized version of the task only identifies similarity up to unitary transforms, and we need to mention that as well. We also agree that testing the general metric ($C \in GL(n)$) is highly relevant and would capture general topological equivalence, even if it is not a proper metric, as far as we know. We will mention this and describe the relevant cases for using each group in DSA. We previously considered an idea similar to your DSA', but found that just comparing the eigenvalues will not work in cases of non-normality. Consider this simple example in which case the pure eigenvalue comparison will fail: https://math.stackexchange.com/questions/1955796/find-two-2-times2-matrices-which-have-the-same-eigenvalues-but-are-not-simila Thank you again for your insightful and thorough comments.
Decision
Accept (poster)