Summary
The paper investigates the emergence of grid cells, known for their hexagonal firing patterns in spatial navigation, using predictive coding—a biologically plausible learning rule. The authors propose that grid cells can be learned by neural networks through predictive coding, which aligns well with the principles of local computations and Hebbian plasticity.
The key contributions are:
- Demonstrating that predictive coding networks (PCNs) can naturally develop grid cell representations with sparse, non-negative constraints, and a temporal extension (tPCN) achieves similar results in dynamic tasks like path integration.
- Establishing that tPCNs approximate the truncated backpropagation through time (BPTT), highlighting a biologically plausible alternative to BPTT for learning grid cells.
- Analyzing the robustness of grid cell emergence in PCNs and tPCNs across varied architectural and environmental conditions, showing grid cells can still form even without velocity input.
Strengths
**Originality:**
This paper provides a new perspective on grid cell formation by applying predictive coding. While previous work has used RNNs trained with BPTT to simulate grid cells, this study introduces predictive coding networks (PCNs) and temporal PCNs (tPCNs) as biologically plausible alternatives. While predictive coding has been addressed in hippocampal formation previously (Stachenfeld et al. ++), the proposed learning rules are novel in this context.
**Quality:**
The authors demonstrate grid cell emergence in PCNs and perform a comparative analysis with existing RNN models. By analytically showing that tPCNs approximate truncated BPTT, they provide a theoretical solid grounding for their approach. Further, the robustness analysis—exploring different model architectures, non-linearities, and environments—addresses shortcomings proclaimed in recent work (Sorscher vs. Schaeffer). The theoretical and empirical sections are well-integrated.
**Clarity:**
The authors use clear visual representations of presented ideas, making interpretation intuitive. The derivations are well-presented, especially in demonstrating the correspondence between tPCNs and truncated BPTT. However, some technical details on the inference dynamics of tPCNs might benefit from additional clarity or simplification, especially for readers less familiar with predictive coding.
**Significance:**
The findings are interesting for neuroscience and machine learning. They suggest that predictive coding may underpin not only perceptual but also spatial and navigational representations. For neuroscience, predictive coding may unify perspectives across cortical functions. For machine learning, it offers an alternative to backpropagation-based learning in dynamic systems.
Weaknesses
Although it is nice to see grid cells emerge in the proposed setup, it is not that surprising given the setup with static place cell readout. The comparison between BPTT and tPCNs is more interesting, in my opinion, than the grid cell results and can have broader implications beyond this particular setting; I would present this as the main result and, therefore, consider moving this result to an earlier stage and presenting the grid cell stuff as a test case.
The model operates under certain assumptions (e.g., reliance on sparsity, non-negative constraints, simplified path integration tasks, and place cell readout) that may not generalize well across different types of neuronal representations or tasks. However, the discussion lacks a critical assessment of these assumptions, specifically regarding where the predictive coding model might fall short compared to other frameworks for grid cells, such as the recent development of self-supervised learning for grid cells ([Schaeffer et al.](https://arxiv.org/abs/2311.02316)), conformal isometry, or distance preservation ([Xu et al.](https://arxiv.org/abs/2210.02684), [Dorell et al.](https://arxiv.org/abs/2209.15563)). For example, the choice of static read-out place cells limits studies of remapping (but can be done; see [Schøyen et al.](https://www.sciencedirect.com/science/article/pii/S258900422302179X), different geometries [Krupic et al.](https://www.nature.com/articles/nature14153) etc.
The proposed predictive coding model successfully generates grid cells, but the mechanistic explanation for how and why grid cells emerge under predictive coding is lacking. Moreover, the field suffers from challenges in comparing representations across studies, barring visual inspection. Grid scores are used to assess grid cell likeness; however, these give little insight beyond 60-degree symmetry. I suggest you use something else to assess the function of the networks, such as ablation studies and studying the full representational setting of the network. For example, do you see border cells, band cells, etc? At least provide examples, preferably representations from the full network, in the supplementary.
All in all, since the title and introduction of the paper highlight grid cells, I would expect more analysis of this finding and a broader comparison with the existing literature. However, I think the more interesting finding is the comparison between BPTT and tPCNs. Therefore, I would recommend lifting this part of the paper and proposing the grid cell story as a potential application motivating further studies on this line of work, although I do see your point on extended analysis on this being out of scope.
Questions
The authors find that grid cells emerge under various configurations and constraints, even in the absence of velocity input. Could they expand on the implications of this finding for the role of predictive coding in spatial learning?
You claim that tPCN approximates tBPTT; however, the RMSE indicates that when the inference has fully converged, the tPCN outperformed tBPTT. Path integration is a Markov process, and it therefore makes sense that tBPTT should work. However, as you show, having the extra inference steps helps. Is it then tPCN that approximates tBPTT or the other way around (tBPTT approximates tPCN)
Moreover, this begs the question: what is the difference between $g_{t-1}$ from RNNs and $\hat{g}_{t-1}$ tPCNs that give this performance boost?
Is there a qualitative difference in grid cells between the models, or are there other cell types that make $\hat{g}_{t-1}$ "better"? One way to hint at this would be to ablate neurons in $g$ and rank them according to their effect on the loss. Are there any differences between these two populations? Another way would be to perform a detailed analysis of the predictive power of $g$ cells in the two models, for example, according to Ouchi et al.
Related works, such as the work from Giocomo in 2011, are outdated. Whether oscillatory dynamics are important for grid cells started as you point out with the work by [Burgess](https://pmc.ncbi.nlm.nih.gov/articles/PMC2678278/) and Hasselmo, but it was later included in CANNs by [Bush and Burgess](https://pubmed.ncbi.nlm.nih.gov/24695724/). The importance of oscillations in grid cells has been tested experimentally by [Lepperød et al](https://www.science.org/doi/full/10.1126/sciadv.abd5684), [Schmidt-Hieber et al.](https://www.nature.com/articles/nn.3340), [Robinson et al.](https://www.sciencedirect.com/science/article/pii/S2211124724009197)
**Minor**
- $\hat{g}$ is used but not introduced as inferred before line 392; this can be nice to point out earlier.
- Whether grid cells are learned or are there from birth is disputed; I would present this in less certain terms.