Summary
This paper frames the learning of consistency models as state value learning in a Markov decision process (MDP), deducing two main performance bottlenecks in consistency training: the variance of the score/noise estimator and the temporal discretization error. The authors then deploy several strategies to tackle these two bottlenecks, notably leveraging a better noise estimator and changing the temporal schedule. The resulting consistency model outperforms the state of the art on standard benchmarks.
Strengths
The idea of **leveraging a noise estimator with lower variance**, taken from the literature of diffusion models, to improve consistency training is interesting. It tackles a well identified problem of consistency models and, based on the presented ablation study, seems to yield valuable results.
Weaknesses
The paper presents too many important flaws, described below, to be considered for ICLR. Therefore, I strongly recommend rejecting the paper.
## Relevance of the MDP framework
The developed MDP framework is **disconnected from the rest of the contributions**. In the paper, the MDP view of consistency models provides no other insight related to the rest of the paper then than the identification of two main improvements directions -- variance reduction and alleviation of discretization errors. However, **this MDP view is not necessary** to identify these directions: it has already been clearly identified in prior work via more direct means.
- The discretization error is already identified in the original paper of Song et al. (2023, Theorem 2).
- Variance is already discussed by Song et al. (2023, Section 5), Song & Dhariwal (2023, Section 3.4) and Dou et al. (2023), for example. The variance problem can directly be identified as originating in the one-sample estimation of the score as already noted by Song et al. (2023, Section 5). It is the starting point of several research papers aiming to reduce this variance (Xu et al., 2023; Pooladian et al., 2023).
Dou et al. A Unified Framework for Consistency Generative Modeling. [ICLR 2024 submission](https://openreview.net/forum?id=Qfqb8ueIdy).\
Pooladian et al. Multisample flow matching: Straightening flows with minibatch couplings. ICML 2023.
## Limited novelty
The proposed solutions to the variance and discretization problems (Section 4) are **orthogonal and either already exist in the literature or are adaptions of existing techniques**. While combinations of existing methods can be acceptable, the contributions should be more developed, better motivated, and further supported by more experiments.
The most valuable novelty is the idea to use an already existing lower-variance estimator of the score/noise (Section 4.1) for consistency training. But this is insufficiently developed and tested. On the same topic, the paper highlights the adaption of this technique to conditional generation, but it is **straightforward**; furthermore, as the authors notice, it is not directly applicable to non-categorical conditioning.
## Unclear writing and unsupported claims
The writing of the paper is lacking in many aspects, even beyond the many typos (cf. next weakness) and leads several claims to be unsupported. **Section 3 is particularly unclear**, hurting the soundness of the MDP view.
- While Eq. (8) might be seen as a Bellman equation in a relevant setting, it is unclear to me why $h_\theta$ is a value function and $r$ a reward, especially as the latter is never optimized anywhere.
- The MDP was only developed for continuous-time consistency distillation at the beginning of the section, but is discussed in other settings (training and discretization) afterwards.
- Eq. (9), (11) and (12) lack context and development to be properly understood. They could be encompassed in small lemmas or propositions.
- The remark on the link with $n$-step temporal difference (line 198) is insufficiently explained.
- The introduction mentions performance bounds (lines 77 to 80), but they are nowhere to be found.
## Lacking finalization
Firstly, given the low significance of the contributions, **the experimental part of the paper deserves more focus**. Besides developing it, I would advice the authors to **release the source code** to ensure the paper's reproducibility.
Secondly, the paper suffers from too many mistakes or typos, making it hard to read, as shown in the following (possibly non-exhaustive) list.
- Mathematical problems:
- $t$ and $r$ are not defined on line 81;
- On line 114, the expectation should be over $x_0$ instead of $x_0 | x_t$;
- On line 136, $t_\lambda$ should be the reverse function of $\lambda_t$, and $\mathbf{\epsilon}(\mathbf{x}\_{t\_\lambda, t})$ should be $\mathbf{\epsilon}(\mathbf{x}\_{t\_\lambda}, t\_\lambda)$;
- In Eq. (7), $\alpha_s$ should be $\alpha_r$.
- Formatting:
- the dataset is not specified for Figure 3's experiment;
- I suspect that most of the figures are not properly readable in grayscale;
- there should be no line between points in Figure 4's plot;
- Figure 5 should be referenced in the corresponding paragraph ("Edge-skipping Multistep Sampling");
- Tables 2 and 3's captions almost overlap;
- lines 523 to 526 should be written using the correct font size;
- it is not specified whether all CIFAR-10 experiments are conditional or unconditional;
- a few references are duplicated (Song et al., 2020; Song et al., 2023).
- Typos/language:
- on line 149: "weighg";
- on line 154, "it is noting" is not correct English;
- on line 188, "depent";
- on line 196, "eough";
- on line 202, "konwn";
- on line 218, "gourd";
- on lines 413 and 415, "LIPIPS";
- on line 523, "a previous papers".