Sequential Memory with Temporal Predictive Coding

Summary

This paper proposes to use Predictive Coding Networks for temporal association of sequences.

Strengths

The paper is well-structured and easy to follow. The motivation is clear: a deep network model with biologically plausible learning algorithms for sequence learning.

Weaknesses

1. This paper lacks of novelty and is in fact a trivial extension of [1]. In [1], the single-layer and two-layer predictive coding networks are proposed to associate the input and the output. In this paper, the input is replaced with frame x[t] in a sequence and the output is replaced with frame x[t+1]. 2. Property 1 in Section 4 is a trivial result for linear regression. It should be noted that Property 1 is not rigorously presented. A condition that the data covariance matrix must be full-ranked should be imposed. 3. Lack of robust retrieval evaluation. In [1] and classic Hopfield networks, the model can recover the stored memories given noisy initialize state. Is the model in this paper robust to noise for sequence storage? Further experimental evaluation is needed. 4. Missing references [2,3]. [2] is the very first work for temporal sequence association. How do the authors compare their work to [3], which is also about predictive coding for sequences? [1] Associative Memories via Predictive Coding. Tommaso Salvatori, Yuhang Song, Yujian Hong, Simon Frieder, Lei Sha, Zhenghua Xu, Rafal Bogacz, Thomas Lukasiewicz. arXiv, 2021. [2] Learning Patterns and Pattern Sequences by Self-Organizing Nets of Threshold Elements. S.-I. Amari. IEEE Transactions on Computers, 1972. [3] Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning. William Lotter, Gabriel Kreiman, David Cox. ICLR, 2017.

Questions

See above.

Rating

3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

2 fair

Contribution

1 poor

Limitations

See above.

Summary

The paper presents work on (relatively) biologically-plausible neural networks for remembering sequences of inputs, extending work on temporal predictive coding nets (a simple architecture of a layer of neurons for feature values and a layer for prediction error, with some interneurons) and asymmetric modern hopfield networks. Analysis shows a direct link between temporal Predictive Coding networks and Asymmetric Hopfield Networks, with improved performance vs AHN for correlated patterns reflecting the implicit whitening process built into the tPC net. The multilayer tPC net develops interesting representations of items and context while solving the problem of storing sequences with repeated items.

Strengths

There are nice clear theoretical results to explain the links between tPC and AHN, and new results showing good performance in challenging sequential memory tasks with complex repeated images. The development of interesting higher order representations of sequential order in this simple-to-analyse system will be of interest to neuroscientists given the development of these representations in the mammalian brain.

Weaknesses

The sequential memory solutions considered here use changes to connection weights to store the sequence, these might be compared with deep networks that are able to reproduce a sequence having been pretrained on similar sequences, but not the one in question (the first example I think is Botvinick & Plaut, Psych Rev, 2006; now transformers). It seems that the only advantage in performance, compared to the AHN, is the whitening process (which could easily be added to AHN), but perhaps this really reflects the fact that this is a biologically plausible implementation of AHN (which introduces whitening as a by product) - perhaps it could be compared to alternative ways of implementing AHN in a biologically plausible way (if they exist)?

Questions

Would an alternative be to explicitly make predictions or learn associations between items that are more than one place apart in the list? How does this model compare to those explicitly involving a sequential contextual signal (e.g. the "temporal context model")?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

See above for potential limitations/comparisons that could be discussed

Summary

The authors propose a temporal predictive coding model that can memorize and recall sequences. The model performs better than a model based on asymmetric Hopfield networks. The authors provide a theoretical evaluation end explain the reasons for better performance. This work is inspired by neuroscience results and the authors argue that it establishes a possible computational mechanism underlying sequential memory in the brain.

Strengths

This paper proposes a new model for learning sequences using temporal predictive coding. The method is well explained, and results consist of several experiments showing better performance than when using an asymmetric Hopfield network. The authors provide a connection between temporal predictive coding and asymmetric Hopfield network - they identified how temporal predictive coding actually performs the same operation as asymmetric Hopfield network but with an implicit statistical whitening step during memory recall. They showed that when using multi-layer temporal predictive coding, the model develops latent representations of contextual information in sequential memories.

Weaknesses

The authors mentioned that this work helps establish a possible computational mechanism underlying sequential memory in the brain. In its current form, the paper lacks direct comparison with neural or behavioral data. Behavioral tasks such as free recall could be used to evaluate if the properties of the sequential memory resemble those in the brain.

Questions

While the authors mentioned some of the related work, here are a few more papers that build neural models of memory for sequences. Graves et al. 2014 Neural Turing machines Voelker et al. 2019 Legendre memory units: Continuous-time representation in recurrent neural networks Eliasmith et al. 2013 A large-scale model of the functioning brain Howard et al. 2014 A unified mathematical framework for coding time, space, and sequences in the hippocampal region Whittington et al. 2020 The Tolman-Eichenbaum machine: unifying space and relational memory through generalization in the hippocampal formation The authors could comment on similarities/differences with respect to these approaches or perform a comparison.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

The authors provided a sentence about future directions, but not about the limitations.

Summary

The paper generalizes predictive coding as a method of training neural networks to Hopfield networks, giving a model of temporal predictive coding (tPC). tPC proves itself able to memorize discrete sequences at a level competitive with Asymmetric Hopfield Networks in experiments, and provides an intriguing hint as to the potential function of statistical whitening in the hippocampus.

Strengths

The authors provide a strong mathematical grounding and linkage to the predictive coding literature. Temporal predictive coding shows the interesting strength that as the correlation between features in a sequence increases, it does not appear to significantly lose much of its capacity -- no doubt due to the implicit statistical whitening. They perform an experimental evaluation against sequential versions of the Modern Hopfield Network and the Modern Continuous Hopfield Network, though not against non-Hopfield sequence learning or cognitive mapping models.

Weaknesses

The authors overclaim about biological/neural "memories" from the first sentence of the abstract. This becomes important because lossless sequence memorization is not what the hippocampus does, and if it did, it would be useless. This leaves the major unaddressed question being: can the paper's tPC model generalize to unseen but reasonably similar sequences? What features of the sequences can change without disrupting memorization?

Questions

Can the authors address any of the literature in the learning of cognitive maps in the hippocampus? Those often provide a connection to discrete event sequences and would give the authors a baseline to compare to beyond just the Hopfield networks they use.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

3 good

Contribution

3 good

Limitations

Are the authors aiming their model and their claims at the computational or algorithmic Marr levels of analysis? Even if they're aiming at the algorithmic level, predictive coding has been proposed to approximate backpropagation in certain limits, and so they could have compared against backprop-trained memory models in which the backprop training was replaced with a sufficient predictive coding architecture. Likewise, there are plenty of cognitive mapping models at the computational model which are not analytically tractable, but which do admit sampling/Monte Carlo implementations that can attain neural plausibility. Why the restricted class of comparisons? Are the authors specifically proposing that the brain performs all necessary computations in closed form? EDIT: The authors have fully addressed my concern about closed-form computation, which then places their tPC model within the class of things to which I requested comparison. This is very good!

Impressive rebuttal, raising my score

To the authors, Thank you for addressing my concerns about this paper, including the few that were a result of my own confusion. The global rebuttal and your specific response here indicates, in my view, a significant strengthening of the paper, and I will be raising my score.