Summary
This paper shows that learning contrastive representations with the infoNCE objective combined with an L2 constraint on these representations permit Gaussian distributions on the representations. The authors then show for timeseries data, that, when we make a specific Markov chain assumption (a kind of linear state-space model) then the distribution over learned representations permits a nice Gaussian form for all future or past representations. As a consequence of this, planning can be performed through simple linear interpolation within the learned space.
The paper shortly validates their method in a didactic experimental setup.
Strengths
This paper is very well written, the presentation of theoretical results, their intuition, and how these are used is for a large part excellent.
The experiments nicely contribute to the main point of the method, that, within a contrastive learned space we can do planning in an effectively closed form rather than using an expensive dynamic programming method. The visualizations nicely show how the authors' method differ from the baselines.
Weaknesses
### Summary
I find the discussion and narrative of this paper too weak to confirm the novelty and contributions of the method. The authors make assumptions on their model such that it satisfies a nice practical property: planning through interpolation. However, the theoretical results that are presented should be well known consequences of Gaussian assumptions for linear state space models. The presentation makes this insight almost seem profound, but the weak discussion of related work makes me skeptical of the contribution.
Although this paper presents a nice observation, taking all listed points of improvement into account, I don't think this paper is sufficient yet for an accept. But I'm open to improve the score if the authors address my concerns.
### Major comments
- Lemma 2 is stated as the main result. Although this way of defining waypoint conditionals is an interesting perspective on planning, the result should be a well-known consequence of using Gaussians (The same goes for Lemma 3). This makes it unclear what the novelty/ contribution of this result is. The authors need to improve this by discussing prior work/ results after introducing their model.
- The title of the paper: inference via interpolation, is too hidden within the main text (at the end of 4.4 in Eq. 7). I would strongly suggest turning this result into a corollary (indeed with proof referring to Newman and Todd 1958 Pg. 471), then moving this result all the way to the top of section 4, and restructuring the text to move towards this result.
- The interpolation in Eq. 7) only presents the mean-representation. There is no such discussion on the covariance.
- The experiment of figure 5 is not well-designed. The introduced method makes use of strong assumptions that allow it to make use of linear-interpolation for planning towards future or past states. Still, the authors compare this to baselines where (also without discussion) we suddenly assume that these have the same functionality, why is this a reasonable assumption? As a result, a baseline without planning outperforms the baselines **with** planning.
- Discussion Line 347 (last sentence) strongly oversells the presentation of the paper. The interpolation property is a nice consequence of the method's assumptions (Eq 3. and As.1, 2) and the contrastive learning. However, there is no discussion on how and why other methods cannot achieve similar things. My suggestion is to drop the 'when' comment, and only keep the 'how', because that's what the paper ultimately shows.
### Technical comments
- Eq 2. is confusing since it is first written that $(x, x^+) \sim p(x, x)$ but in the Eq. it appears as $p(x)$. The term $p(x, x)$ is wrong, write this as $p(x, x^+)$ and keep its use consistent. Perhaps use $(x^+, x) \sim p(x^+ | x)p(x)$ and $x^- \sim p(x)$?
- Assumption 2, phrasing this result as an assumption weakens the story of the authors, especially since this is the equation that the authors' rely on for later analysis. It's usual to make assumptions or simplifications of the real world, so the fact that approximation error or model misspecification might not satisfy the result should not take away from it. I suggest making this a proposition and writing an assumption around this that discusses approximation error, please directly refer to the proof afterwards (citation + where to find it).
- Lemma 3, $\eta$ isn't clearly shown to have zero entries apart from the first and last index. Why not use the same definition as in Appendix A.4?
- Missing proof of Lemma 3, no reference to appendix.
### Minor comments
- Line 166 typo: 'that that'
- The two representations are $\phi$ and $\psi$ composed with $A$. Calling this two separate representations is confusing, there is only 1 learned representation and $A$ is used to learn the transport from embedding $\psi(a)$ to $\psi(b)$. Maybe state this upfront, i.e., in the first sentence line 187 say $\psi(x)$ and $\phi(x) = A\psi(x)$, that helps when reading chronologically.
- Line 215, refer back to Eq.3 for the definition of $c$
- Line 231, proof of lemma 2 is in appendix A.3, not A.2.
- The three examples in 4.3 for lemma 2 are nice to have for an unfamiliar audience, but I don't see how this contributes to the result since $A$ is learned, and will likely never satisfy trivial solutions.
- Line 268, the interpolation result is nice, but the linear interpolation insight is not particularly profound (the text makes it seem as if it were so).
- The start and end symbols in Figure 4 are too small to see and overlap with the black dots too much, even at 500% zoom.
- Figures 5, 6 are not chronologically ordered/ numbered (6 appears before 5).
- The choice of baselines and experiments are somewhat limited. I would have liked a comparison to a decoding method which the authors criticize in the introduction paragraph 2 (e.g., using the DreamerV2/ PlaNet RSSM model).
- There is still space left on page 9, the authors could have spent more space on discussing the limitations of their method, which right now is rather shallow.
Limitations
1) the paper concludes that it provides a kind of 'recipe' for how to get models that have a linear interpolation property for planning, but aside from the model that the authors designed in a very specific way this is not discussed.
2) the experiments are somewhat trivial since the authors expected baselines, that were not designed for a particular feature (planning through linear interp.) to suddenly have this feature.