Self-Supervised Grid Cells Without Path Integration

Reply

On a general note: thank you for your kind feedback. On presentation: Thank you for raising this point: We agree, and will add subtitles to each result. On comparisons with other work: As we write in response to other reviewers, we will add a discussion on how our work relates to other models, and the role of L1 and L2 capacity constraints. The main motivation for utilizing these two terms, we argue, is that (similar to Dorrell et al., and Xu et al.) spatial representations should accurately reflect distances in a neighborhood around your current location, and that representations should be efficient (using as little of the state space as possible, similar to Schaeffer et al.), and distributed, which also affords robustness. Otherwise, we would highlight simplicity as one of our main modeling motivations, as a huge range of models with multitudes of different regularization strategies, constraints and architectures all report grid-like representations. Regarding path invariance: This might be implicitly enforced by (non-infinitesimal) distance preservation in our model: If an RNN approaches a location from different directions, then the distance between representations should only depend on spatial distance (not incoming direction). However, this could also be the case for the CI loss formulation in Schaeffer et al., which could point to the path invariance-term being somewhat redundant, at least in that model. However, it could be that path invariance is required for learning consistent representations in that model, due to e.g. long sequences (whereas CI is only required for short distances). Due to the many interacting loss terms, we are unsure about the exact contribution of the L2 capacity constraint in Schaeffer et al.; but they suggest it is linked to multimodularity, and spatial tuning. They do report that ablating separation loss causes loss of spatial tuning, so it might be that their scale separation term forces similar representations to be aligned (maximizing differences between distinct states), inducing distributed representations, which we find can be introduced directly by an L1 constraint. We will, however, discuss this more in the revised paper. Concerning the choice of model: There are several reasons why we employ our model, instead of others. The first, as mentioned, is simplicity: comparing to e.g. Schaeffer, our model features just two loss terms, a vanilla RNN architecture (compared to an RNN where the weights are a velocity-dependent MLP), and we observe that the RNN only features one module of grid cells. The second is that the same objective is directly “portable” to non-path integrating networks (i.e. a feedforward network), which are key to demonstrating that path integration is not necessary for grid emergence. It is unclear how that could be done using Schaeffer et al.’s model, which requires path integration e.g. to define path invariance. On speed: This is a great suggestion, and we will add examples of models trained with varying trajectory simulation speeds. In particular (as was also asked by another reviewer), we are interested in what happens when the speed goes to zero, and we will add higher-speed agents. Notably, zero-velocity RNNs do not exhibit bands, similar to the feedforward network, potentially ruling out architectural differences as the cause of these patterns. On Fig. 2d: thanks for pointing this out, we will improve upon the caption, and the figure legibility. It shows the path integration error at every timestep, for subsamplings (of size n = 29 units) of the RNN population. It demonstrates that there is a linear relationship between error and (subpopulation) grid score at every timestep, showing that grid cells (high grid score units) are consistently less important than band cells, and apparently that errors compound when pruning low grid score units, indicating that path integration has been disrupted. On Fig 3c: Thanks for catching this. We will correct it.

Revision

We have added several (supplementary) analyses to address your concerns to the revised text, including - subtitles for clarity - A longer comparison between our model and others (Appendix A4) - An analysis of the speed dependence of learned representations, showing that RNNs trained with zero-speed trajectories do not learn bands. Also, for larger speeds, band cells become more prominent, and patterns more square. (Appendix A5) - Added analyses that showcase the effects of model parameters (Appendix A9, A6, A8) - Added a linear decoding scheme as an alternative analysis of path integration ability, and showcasing that path integration ability correlates with band tuning in the representation during training. (Appendix A7) - improved captions and removed existing typos. - Highlighted a potential difference between a CI requirement and our distance preservation (Appendix A9) Together, we hope that these improvements meet your concerns. Please reach out if you have more questions.

Reply

We thank the reviewer for their perspective. While our strategy for arriving at grid-like representations is not new, and builds upon Dorrell et al., Xu. et al. (2022) and Schaeffer et al. (2023) (which we reference when introducing the model), we argue that our main modeling novelty, is producing a highly simplified model of single grid module emergence, using only two distinct loss terms. Our main novelty, however, is applying such self-supervised methods to tackling the problem of whether grid cells do path integration. In comparison, Sorscher et al. and Schøyen et al. utilize a model tasked with predicting place cell activity that exhibits a range of emergent cell types. In our case, our model does not require any place cell labels, and only exhibits two distinct cell types: Grid cells, and, if path integration is present, additional band cells. It is this separation into only two cell types that allows us to perform functional assignments, even in the self-supervised setting. We would also like to highlight our modeling and pruning approach as an important novelty in two ways. First, we prune velocity projections to the recurrent units compared to Schøyen et al. and Nayebi et al who ablate recurrent units. This pruning strategy has the advantage of not confounding recurrent homeostasis effects with path integration performance. Second, we evaluate path integration on the recurrent representation as it mitigates the possibility that path integration disruption is confounded by disruption of the readout into place-like labels present in the model of Sorscher et al. We will be happy to add this as part of the discussion of our paper. Regarding that the model only exhibits a single module: We agree that biological grid cells arrange in multiple modules with differing orientations and scales. However, we want our model to be highly simplified, precisely to uncover questions of path integration within modules. As we show in Fig. A1, however, the scale of the learned representation depends on the scale parameter sigma. Thus, multi-modular grid cells can be introduced by partitioning the network into different scales, similar to Xu et al. (2022). If less control of the learned representations is desired, more strongly self-supervised objectives is likely required, akin to Schaffer et al. (2023). On the choice of loss function: We thank the reviewer for raising this concern: The key property is that distances should be preserved in a neighborhood around your current location, a point which was also raised by Dorrell et al. (2022). Regarding why isometric mapping is restricted to the local environment, although this property is evaluated locally, its required everywhere in the environment (expectation over spatial positions in Eq. 1), making it a global property. Hence distance preservation is preserved everywhere, e.g. (global) distances computed through integration. And we agree that this is likely what induces unimodular grid patterns - because we dictate the scale at which distances should be preserved. As such, it could be interesting to demand e.g. scale-invariance/equivariance, or directly introduce multiple modules in our modules by partitioning the network into discrete scales. As previously described, however, we think that unimodularity is useful in our model, as it makes the resulting network simple, and easy to probe for path integration. On comparison with other models; thank you for raising this point; as also requested by other reviewers, we will add a discussion comparing our model to other recent normative models. And while we do reference Sorscher et al., we do not consider a thorough comparison between our model and Sorscher’s as relevant as comparing to the more recent and similar models of Xu et al. and Schaeffer et al, upon which our work is chiefly based. But key differences include (i) imposing path integration directly on the recurrent representations (similar to Schaeffer et al. 2023) compared to Sorscher’s place-cell readout (which has been critiqued in e.g. Schaeffer et al. 2022) and RNN-initialisation mechanism, (ii) our model features unit-normalization (iii), our model uses a L1 capacity constraint rather than L2 weight decay (iv). We will also include ratemaps of all units present in our models in the appendix - showing that our model cell types is limited to grid-like cells and band cells (for the RNN model), we realize that this did not come across clearly before - thanks again.

Revision

Besides the points mentioned above, we have included several additional analyses in the revised manuscript (as appendices), including - A comparison of our model and others (Appendix A4) - Ratemaps of all units in both networks, demonstrating that all units are either band- or grid-like. (Appendix A6) - Higlight a distinction between our distance preservation and a CI requirement (Appendix A9) We hope this answers your questions, please reach out if you have other concerns.

Reply

Thank you for your insightful comments. While it is true that other RNN models perform path integration using only grid-like units, it is also the case that these models use a non-standard recurrent architecture, wherein the recurrent weight matrix is itself a neural network, dependent on the incoming velocity input. In contrast, neurons in our network are computed using a standard weighted summation of velocity and recurrent inputs. We hold that, while neither model is particularly biologically plausible, our model is simpler, and at least closer to standard models of recurrent models of neuron information integration. We will update our discussion to reflect this position. On the usage of short trajectories: We train on short sequences to simplify training (shorter sequences allow for more parallelized computation making these models faster to train) and mitigate vanishing/exploding gradients. Additionally, in its simplest form, path integration is a Markovian process, and so only short trajectories should be required to learn perfect path integration. Whether this is biologically plausible is, on the other hand, a fair critique. However, we did evaluate our model on long sequences (shown in Fig. 3c)), which demonstrate that grid-like representations persist for long sequences, even generalizing outside the training environment. We realize, however, that this does not come across clearly from either the main text or caption, so we will update both. On Dordekt et al.: thank you for raising this point; we will refer to Dordek et al. in the text. The major difference between our work and theirs, is that they do PCA on simulated place cell representations, learning that grid cells can act as a low-dimensional representation of place cells. Our (feedforward) model, on the other hand, and the representations learned by it are solely defined by the distance-preservation/capacity task. Regarding the possibility of other unit types in the feedforward network: The answer is no: While some units exhibit less pronounced hexagonality, they all do so to some extent. We can, for instance, include ratemaps of every network unit for both networks, in a supplementary figure. On path integration ability: While we use the state distance (between pruned and unpruned networks) to gauge path integration ability in a decoder-agnostic way, we realize that we do not clearly demonstrate that the RNN is capable of path integration, beyond demonstrating that the network produces consistent ratemaps (for which path integration is required). We will therefore train a linear decoder (as we have observed that linear decoders sometimes struggle to accurately decode locations), and include a population decoding strategy, and demonstrate this capability explicitly - thank you for raising this point. On band emergence: Correct - the RNN is initialized using random weights, and the resulting initial representations are indeed random. We will add a supplementary analysis of path integration ability as a function of training time, and correlate this with band/grid scores of network units. On non-grid networks: Again, this is an excellent point. We will train additional networks with lower/higher capacity (which strongly influences learned representations) and see whether band-like representations are present, as this might point to bands as a general mechanism for path integration.

Revision

Thank you for your patience. In the current revised version, we include several analyses (appendices) that we hope will answer your questions, including - evaluating the model on long sequences (Appendix A7) - Training a modified linear decoder to gauge decodability (Appendix A7) - Included Dordek et al. in a comparison of our model with others (Appendix A4) - included full ratemaps of all units in RNN and FF networks, demonstrating that all units are grid-like (Appendix A6) - Included a band-analysis as a function of training time, demonstrating that more strongly band-like representation correlate with increased path integration ability during training. (Appendix A7) - Train a low-capacity RNN, and demonstrate that it does not have clear band-like representations, while still learning consistent spatial representations (which requires path integration). (Appendix A6) We hope this answers your most pressing questions, thanks again for your input.

Reply

In general, we would like to thank you for your helpful feedback. Regarding the comparison between our work and related models, including those by Xu et al. and Schaeffer et al., we will include an expanded discussion in our manuscript. This discussion will directly compare our approach to these models, highlighting both similarities and differences to provide a clearer context for our contributions. We will also add a discussion and study on L1 vs. L2 capacity constraints, which also adds to the discussion on our model vs. others. Also, we will add a supplementary figure that shows learned representations as a function of regularization norm. On band cells: We will refer to cueva et al., and refer explicitly to experimental result concerning band cells. For transparency, we will also make sure to refer to the controversy surrounding band cells (e.g. Navratilova et al., 2016; Grids from bands, or bands from grids? An examination of the effects of single unit contamination on grid cell firing fields) Regarding the description of the inputs to the model, this is described at the start of the methods section. We acknowledge that this could be made clearer - thank you for pointing this out. To improve clarity, we have rewritten it, and included standard mathematical notation to supplement the textual description of the inputs to each network. Regarding effects of hyperparameters: We understand that the importance of each hyperparameter should be presented more clearly. We do perform a hyperparameter search in terms of the loss weighting parameter alpha, and the Gaussian tuning scale sigma, located in the appendix. Although this was mentioned in Section 3.2, we had not included a reference to the appendix. This will be included in the updated manuscript and discussed more clearly in the revised version. Concerning the title: This is a great point, and the reason we use the wording “without path integration” is two-fold: First, we find that grid-like representations emerge in feedforward networks that do not path integrate. Second, in recurrent networks, we find that we can remove velocity input to grid cells, with minimal impact on path integration performance. However, we agree that band-type units could still arise from the architectural difference between networks. We have therefore trained recurrent networks on datasets with different velocities, ranging from 0 velocity, to five times that used in the other models used in this work. Notably, band units do not emerge in the RNN that is trained with zero velocity input. We will add this figure to the supplementary, and update the results section to reflect this - thank you. On orientations of grid units: It is true that we observe a range of orientations in the feedforward network, and that there seems to be two dominant orientations, which could be consistent with modularity in biological grid cells (see e.g. Stensola et al., 2012, The entorhinal grid map is discretized). However, for the RNN, we observe only a single orientation and module, with minor variations, which is seen in biological cells (see e.g. Redman et al. 2024, Robust variability of grid cell properties within individual grid modules enhances encoding of local space). We will discuss the discrepancy between models in more detail in the updated text. Concerning Fig. 1d: Thank you for pointing this out. As results are purely simulated, units of grid spacing are arbitrary, and relative to the environment size (described in the methods section), while the orientation is given in radians. We will update the figure to reflect this.

Update

We have now added several appendices, that we hope will meet your concerns, including - A substantial discussion on our work in relation to other models (appendix A4) - Demonstrations of L2 capacity constraints (Appendix A8) - Made improvements to text and citations for legibility - We have updated the model and hyperparameter search description - We have added an analysis of the scale parameter sigma to showcase its contribution (and trained models with other alpha values) (Appendix A9) - We have made trained models with varying speed distributions. Notably, for no velocity, band-like representations vanish in the RNN. Conversely, for large velocities, bands become more numerous. (Appendix A5) We hope this answers your questions, please reach out if you have other concerns.