Summary
This paper presents a universal predictor-corrector method for faster sampling of diffusion models, with model retraining. The key idea is to further include the current point along with previous $p$ points while estimating the data point by adding a correction step. It shows that this model can achieve an order of accuracy $p+1$. Experiments on several image datasets show that this proposed method achieves strong performance in comparison to prior SOTA methods.
Weaknesses
**Limited novelty and design**:
- The idea proposed here is marginal in the sense that at each step, the current estimate is additionally used along with some previous points (the latte is the same as [25,34,40]); Given the sequential nature, the current point is actually already used by previous methods [25,34,40] in the next steps (with a step delay), whilst it is just that this point is used for more times. No extra information is used whilst also no clear understand on why this extra use of the current point can lead to increase in the order of accuracy.
- Besides, in general, there are two families of diffusion models: DDPM (e.g., [A, 34]) and SGM (e.g., [35]). And the DDPM series are often easier to speed up due to adopting a variance preserving (VP) process, in contrast to SGM’s variance exploding (VE) process. It is not clear which family this proposed method is focused on, or both?
- It looks the step of 10 is a milestone. Is there any relationship or theoretical implication of this proposed method on tackling this issue?
- Also, the addition operations introduce some complex parameter, which adds further burden to the model tuning process. It is unclear if these parameters are general for different datasets.
**Limited results gain**
- Whilst the numeral results such as FID looks good in comparison, when it comes to the visual examples in supplementary, the generated images are of similar quality as previous methods such as DPM-Solver++. It is known that numerical metrics are some limited in the visual perception evaluation. Overall, the generation performance gain is some limited, not as convincing as what claimed.
- It is some inconsistent when evaluating UniC/UniP/UniPC. How is this selection done?
- If I am not missing, there is no exact evaluation on the benefit of using the current point in estimation, which is the key design of the whole model.
**References**:
- [A] Jonathan et al. Denoising Diffusion Probabilistic Models. NeurIPS 2020
Rating
3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.