Learning Time-Invariant Representations for Individual Neurons from Population Dynamics

Summary

This paper introduces NeuPRINT, an unsupervised approach that simultaneously learn (i )a dynamical model of recorded neural activity and (ii) a time invariant embedding of individual neurons using transformers. Using a downstream transcriptomic classification task, the authors show that the learnt embedding space contains relevant information about the neurons identity, and demonstrated improved performances against a range of baseline methods.

Strengths

The proposed method not only allows to model the dynamical systems governing individual neurons recorded activity, but it also links stereotyped temporal motifs to time invariant location in an embedding space. The latter can therefore be interpreted as a representation of the intrinsic properties of the recorded neurons, like their conductance or capacitance. The approach is conceptually interesting and could have interesting application in characterizing neural population from temporal traces. The authors designed an original model that uses summary of population activity as input to each neurons. The approach is adapted on a range of architecture, and demonstrates that one can leverage the flexibility of transformers to find embedding more informative about neural identity than with traditional model like linear dynamical system.

Weaknesses

The method description is not detailed enough and does not allow to reproduce the results presented in the paper. Overall, it is not very well written. 1) The statistics used to build $C_t$ and $S_t$ are never defined. Do they correspond to 1st and 2nd moment of calcium traces or to more complicated statistics ? 2) Only pseudo-code is provided. 3) Almost half a page is dedicated to Hodgkin-Huxley equations, which have no technical value in the present paper. Surely it would be possible to link the time invariant neural embedding with intrinsic properties of the neurons (like its shape, conductance, capacitance etc.) without introducing many variables that are never used subsequently. In contrast, very little is said about the new method itself. 4) Section 4.2: without further explication, one cannot understand the difference between the two losses used. "Negative Log Likelihood of Gaussian" (NLE) and Mean Squared Error (MSE) are exactly the same thing if the variance of the former model is 1. If the variance is not modeled/learned, then they only differ from multiplicative and additive constant. The fact that the authors did not find a monotonic relationship between MSE and NLE in Table 2 could mean two things: that (i) they indeed used different loses (in which case the method description should be more detailed), or (ii) that the same loss was used in both columns and that the difference in performances simply results from the use of different random seeds. 5) Section 1.1 "Related work" needs rewriting. NeuPRINT is interesting in that it combines several feature of well known neural data analysis techniques. For example, the use of summary statistics of population activity $P_t$ echoes many latent models (see Saxena 2019). Yet, contrary to standard techniques like GPFA, the influence of the population on individual is captured by equation (3), which relates the NeuPRINT to coupled model like (Pillow 2008 or Truccolo 2004). In its current state, section 1.1 simply list existing approaches without really relating them to proposed approach. Minor Points: 1) The vocabulary is often vague (*e.g.* "numerous insights into how the brain works", line 20) and many abreviations like "w/" or "w/o" are unecessarily used in the main text. 2) I agree that the term "representation" is conveniently overloaded but the paper would gain in clarity by attempting to better explicit it. In the present study, from my understanding, the representation of a neuron is an "embedding of its time-invariant properties that have an effect on its dynamics".

Questions

1) How are $C$, $S$ precisely defined ? (at least in the experiment) 2) The author only evaluate their model using the discovered embedding space. What about the dynamics $f$ ? How do the calcium traces reconstructed by the model compare with linear dynamical systems for example ? 3) Why is the dependence on $i$ dropped from equation (3) to equation (4) ? Although $P$ must be permutation invariant, it might be relevant to model slightly different input $P^{-i}$ for each neuron. Since it does not seem much more computationally expendive to do so, can the author motivate this modeling choice ? 4) How is the embedding dimension chosen ? Can it be interpreted like other method like t-SNE or UMAP ? 5) Would it be possible to jointly learn $f$, $\Phi$ *and* $P_t$ ? In doing so, the model would yield interpretable embedding and neural trajectories !

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

The authors adequately addressed the ethical limitations of their paper.

Summary

In this manuscript the authors describe an approach to predicting the transcriptomic identity of a neuron from its activity dynamics that they call NeuPRINT. Specifically, the authors take a dataset with calcium imaging and transcriptomic identification of the imaged neurons. First, they train up a model – either linear, nonlinear, recurrent, or a transformer – to predict the next timestep of the calcium trace using both past activity and what they term "time-invariant" parameters. Next, a downstream classifier is trained to use the learned parameters of the model to predict the genetic identity of each neuron. The authors demonstrate that their model outperforms the current state-of-the-art along with a series of control baselines.

Strengths

Strengths: 1. Their model includes some interesting ideas. Namely, the use of population activity and time-invariant parameters seems to confer substantial predictability. 1. The performance of the model appears to be impressive on this multi-modal dataset. 1. Per the ablation study shown in Figure 3, the model appears to make use of all features that were fed to it.

Weaknesses

Weakness. Major: 1. A major claim made by the authors is that their method outperforms both LOLCAT and simple, effectively random baselines. While their model does appear to perform better, this does not look like an apples to apples comparison. 1. First, it would be appear that LOLCAT uses *only* a neuron's activity in order to classify its transcriptomic identity. On the other hand, NeuPRINT includes activity from nearby neurons in addition to behavioral covariates such as pupil size and running speed. Unless I am mistaken, performance comparisons between NeuPRINT and LOLCAT with similar inputs (single neuron activity *only*) are not shown until the ablation study in Figure 3. Figure 3 right shows performance that appears to be similar to LOLCAT. Is the boost in performance with NeuPRINT solely coming from the population activity and behavioral covariates? If so, then authors would need to scale back their claims about superior performance. Alternatively, the authors could add this information to LOLCAT for a more fair comparison. 1. Second, I am a little confused about the PCA/UMAP baselines. Here the authors selected a *random* window of time to project the data down to 64 components. Why 64? Did this account for a specific amount of variance? Why a random window of time? How was the window size chosen, and why not use a sliding window approach or use the full time-series? 1. Whether or not NeuPRINT outperforms LOLCAT, the LOLCAT paper systematically interrogates their model for insight into the underlying biology of neural systems. I could not find any comparable biological insight here. Minor: 1. The discussion of time-variant and time-invariant is fairly confusing. For example, in line 233 in the description of LOLCAT, it is asserted that "As a consequence of the supervised training scheme, LOLCAT does not extract time-invariant representations of neuronal identity and its perfomance is constrained by the number of labels available in the dataset." Are the authors saying that ISI distributions are time-variant? Also, why wouldn't the performance of *any* classifier be limited to the available labels? I see no reason why this wouldn't also apply to NeuPRINT. 1. I found a number of typos, the manuscript needs a copy-edit.

Questions

1. In Table 1 does the w/o population column mean NeuPRINT *w/o population or behavioral covariates*? In other words, does NeuPRINT have access to information that LOLCAT does not? 1. Can the authors interrogate the learned parameters of their model to learn anything new about the dataset they are modeling?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

2 fair

Contribution

3 good

Limitations

The limitations of NeuPRINT could receive a more thorough treatment (see points above).

Summary

Understanding neural dynamics is crucial for studying brain function, but most existing methods assume that all neurons are functionally identical, which is not the case at the biological level. The biological variability among neurons can influence their dynamics, and since this variability is genetically encoded, the dynamics associated with neuron types would be time-invariant. This work introduces a method that uses transformers to learn a time-invariant representation of neural dynamics. The learned representation can reconstruct neural activity more accurately than existing methods when applied to neural recordings in mice. The authors also demonstrate that the genetic identity of the cell can be captured more robustly from this representation compared to alternative methods and lesion versions of the model.

Strengths

The paper is adequately written and technically good. The method was tested and shown to work well when applied to calcium neural recordings in mice. The introduced method utilizes transformers to identify time-invariant dynamics associated with neural responses and relates them to the genetic heterogeneity observed in neural populations. Understanding the interplay between genetics and other factors, such as learning or external stimuli, in their relationship to neural dynamics is critical for studying neural function, and this work successfully extracts specific dynamics that can help predict neural type.

Weaknesses

While the authors demonstrate how their method can extract neural dynamics related to neuron type, it is important to note that the dataset used only contains spontaneous neural activity. Neural dynamics are typically associated with specific computations and not just baseline states. Therefore, extracting time-invariant or baseline dynamics is relatively straightforward, and the reconstruction analysis may have limited significance. A more interesting application of these methods would be to disentangle dynamics associated with time-invariant or genetic factors from those associated with task variables when animals are engaged in a task or exposed to stimuli. In terms of identifying neural type in the supervised learning task, the authors compare the classification performance of their method to other methods using neural dynamics. However, they overlook alternative approaches based on multiple neural properties such as firing rates, autocorrelograms, or morphology. If the goal of the introduced method is to label subpopulations or neurons, a direct comparison with commonly used approaches to putatively label neurons would be necessary to fully assess the significance of the method. Moreover, in the context of calcium imaging, the expression of the calcium indicator is typically given by genetic markers, so the type or types of neurons being monitored are known. Incorporating this explicit knowledge of neuron type when extracting dynamics could improve the accuracy of the method. The accuracy results demonstrate that the proposed method can identify neuron types better than alternative methods. However, the overall accuracy is relatively low and fails to identify less prevalent neuron types. This suggests that the practical utility of the method for neuroscience research may be limited. Providing information on the computational and data demands of the method, as well as its applicability to other datasets, would be important to assess its relevance.

Questions

Can the approach be directly used on other kinds of neural recordings like electrophysiological data?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

The authors should include a section or clear description of the limitations and assumptions of the method. This section would also benefit from including information on the computational cost, training time, and data demands associated with the method.

Summary

The authors evaluate several approaches for identifying neural types and subtypes (e.g., GABAergic cells are commonly divided into several subclasses Lamp5, Pvalb, Vip, Sncg, Sst). They used a dataset from a recently (2022) published neuroscience paper and compared their approaches to a recent study that had the same objective. To identify neural types, they use several 2-photon calcium imaging recordings (that provide a proxy for neural activity) from mice V1 and behavioral information (running speed and pupil size). They used mRNA expression for 72 genes to label the neurons according to the class (this was then used to compute the accuracy). While performance for class accuracy was relatively similar for all methods, the authors concluded that an approach based on the "Attention is all you need" transformer led to the best results classification results for subclass prediction. They evaluated the contribution of different factors of the model (e.g., running speed and pupil size) and concluded that each contributed to the classification accuracy.

Strengths

The authors compared several approaches and provided detailed information about parameters in the supplemental material. They conducted an ablation study by evaluating the contribution of different pieces of data. The results are presented in a clear form, illustrating the overall accuracy as well as confusion matrices. The proposed approach led to better accuracy than in recent (2023) previous work that used the same dataset.

Weaknesses

The evaluation was limited to a single dataset. While, as the authors state in the discussion, this is an initial attempt to tackle this problem, it remains unclear whether the results would substantially improve if the authors did a hyperparameter sweep or used a more powerful transformer architecture.

Questions

While the authors compared several approaches and concluded that the one based on transformers led to the best results, I wonder if they conducted hyperparameter sweeps to evaluate that the difference in the results between different methods was not because of different choices of a learning rate. While table S3 was very useful for examining the hyperparameters that were used, it would be helpful to provide explicitly the number of trainable parameters used in each approach. The authors used transformer architecture from "Attention is all you need" paper. Would more recent transformed architectures lead to better performance? Was there a reason to use only 2 attention heads? What was the robustness of the approach? For example, if you run the code twice with different seeds, how many cells would change the class/subclass? Did different methods provide somewhat complementary results (e.g. were some neurons correctly labeled with recurrent model and incorrectly with transformer)? If yes, would it be possible to use some form of ensemble learning to get an even better performance?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

The paper addresses limitations (although that could be expanded) and has a broader impact section.