UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models

Diffusion probabilistic models (DPMs) have demonstrated a very promising ability in high-resolution image synthesis. However, sampling from a pre-trained DPM is time-consuming due to the multiple evaluations of the denoising network, making it more and more important to accelerate the sampling of DPMs. Despite recent progress in designing fast samplers, existing methods still cannot generate satisfying images in many applications where fewer steps (e.g., $<$10) are favored. In this paper, we develop a unified corrector (UniC) that can be applied after any existing DPM sampler to increase the order of accuracy without extra model evaluations, and derive a unified predictor (UniP) that supports arbitrary order as a byproduct. Combining UniP and UniC, we propose a unified predictor-corrector framework called UniPC for the fast sampling of DPMs, which has a unified analytical form for any order and can significantly improve the sampling quality over previous methods, especially in extremely few steps. We evaluate our methods through extensive experiments including both unconditional and conditional sampling using pixel-space and latent-space DPMs. Our UniPC can achieve 3.87 FID on CIFAR10 (unconditional) and 7.51 FID on ImageNet 256$\times$256 (conditional) with only 10 function evaluations. Code is available at https://github.com/wl-zhao/UniPC.

Paper

References (47)

Scroll for more · 35 remaining

Similar papers

Peer review

Reviewer xuvS7/10 · confidence 2/52023-07-05

Summary

This paper propose a predictor-corrector method to accelerate the diffusion sampling process, where a corrector is proposed to correct the initial estimation of x_t using previous and current points. The experiments are conducted on imagenet and cifar10 both of which outperform existing efficient sampler at very few sampling steps.

Strengths

1. Improving existing sampling method by a unified predictor-corrector solver is reasonable. 2. Adequate theoretical and empirical analysis of the proposed methods. 3. The paper is well organized and easy to understand.

Weaknesses

N/A

Questions

N/A

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The authors adequately addressed the limitations

Reviewer 5Qcj5/10 · confidence 4/52023-07-05

Summary

This paper develops a unified corrector (UniC) that can be applied after any existing DPM sampler to increase the order of accuracy without extra model evaluations, and derive a unified predictor (UniP) that supports arbitrary order as a byproduct. Combining UniP and UniC, a unified predictor-corrector framework called UniPC for the fast sampling of DPMs has a unified analytical form for any order and can significantly improve the sampling quality over previous methods, especially in extremely few steps.

Strengths

On both unconditional and conditional sampling using pixel-space and latent-space DPMs. UniPC can achieve 3.87 FID on CIFAR10 (unconditional) and 7.51 FID on ImageNet 256 256 (conditional) with only 10 function evaluations.

Weaknesses

They claim many times on unified or model-agnostic but no experiments support their claim, the method only tested on DPM. Also the writing make me lost many times when reading the paper, I suggest you rewrite your motivation part by which technical issue in DPM you try to solve, why your method works. The theory on faster convergence property is also confused on accuracy, is it really useful? how to theoretically guarantee on the accuracy.

Questions

They claim many times on unified or model-agnostic but no experiments support their claim, the method only tested on DPM. Also the writing make me lost many times when reading the paper, I suggest you rewrite your motivation part by which technical issue in DPM you try to solve, why your method works. The theory on faster convergence property is also confused on accuracy, is it really useful? how to theoretically guarantee on the accuracy.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

2 fair

Contribution

2 fair

Limitations

See Weaknesses.

Reviewer TtDL3/10 · confidence 4/52023-07-07

Summary

This paper presents a universal predictor-corrector method for faster sampling of diffusion models, with model retraining. The key idea is to further include the current point along with previous $p$ points while estimating the data point by adding a correction step. It shows that this model can achieve an order of accuracy $p+1$. Experiments on several image datasets show that this proposed method achieves strong performance in comparison to prior SOTA methods.

Strengths

The paper is generally written clearly The problem of accelerating diffusion model is critical to solve Reasonably good performance achieved

Weaknesses

**Limited novelty and design**: - The idea proposed here is marginal in the sense that at each step, the current estimate is additionally used along with some previous points (the latte is the same as [25,34,40]); Given the sequential nature, the current point is actually already used by previous methods [25,34,40] in the next steps (with a step delay), whilst it is just that this point is used for more times. No extra information is used whilst also no clear understand on why this extra use of the current point can lead to increase in the order of accuracy. - Besides, in general, there are two families of diffusion models: DDPM (e.g., [A, 34]) and SGM (e.g., [35]). And the DDPM series are often easier to speed up due to adopting a variance preserving (VP) process, in contrast to SGM’s variance exploding (VE) process. It is not clear which family this proposed method is focused on, or both? - It looks the step of 10 is a milestone. Is there any relationship or theoretical implication of this proposed method on tackling this issue? - Also, the addition operations introduce some complex parameter, which adds further burden to the model tuning process. It is unclear if these parameters are general for different datasets. **Limited results gain** - Whilst the numeral results such as FID looks good in comparison, when it comes to the visual examples in supplementary, the generated images are of similar quality as previous methods such as DPM-Solver++. It is known that numerical metrics are some limited in the visual perception evaluation. Overall, the generation performance gain is some limited, not as convincing as what claimed. - It is some inconsistent when evaluating UniC/UniP/UniPC. How is this selection done? - If I am not missing, there is no exact evaluation on the benefit of using the current point in estimation, which is the key design of the whole model. **References**: - [A] Jonathan et al. Denoising Diffusion Probabilistic Models. NeurIPS 2020

Questions

Please see the weaknesses above

Rating

3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

2 fair

Contribution

2 fair

Limitations

N/A

Area Chair t2zM2023-08-18

Could you look at the authors' response?

Dear reviewer: Please look at the authors' response and other reviewer' comments. Thanks!

Reviewer vcSP5/10 · confidence 3/52023-07-10

Summary

In this paper, the authors present a novel sampling solver called UniPC for diffusion models. UniPC consists of two parts: UniC and UniP, where UniC corrects the estimation with prediction of current timestep and it can be applied other sampling solvers and UniP is a special case of UniC and share the similar form for sampling. Compared with previous methods(e.g. DPMSolver), empirical evaluation have verified the effectiveness of UniPC in generating better results with less sampling steps.

Strengths

- The proposed method can significantly reduce the number of sampling steps to less than 10, without losing the quality of the results. - The proposed method has some generality and can further improve the approximate accuracy of other existing DPM samplers.

Weaknesses

- Experiments. Please conduct experiments on datasets with larger resolution images to measure inference time, quality, and diversity. - Recent work: Please add more details on the related wok about DPM solver and some related ODE-based sampling solver.

Questions

See weakness.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

See weakness.

Reviewer uABf6/10 · confidence 3/52023-07-10

Summary

This paper proposes an ODE sampler for diffusion probabilistic models (DPM), exploiting the structure of exponential integrators. The paper claims the proposed sampler can use any existing DPM and achieve high sampling quality with very few (<10) number of function evaluations (NFE), and also improve upon related methods when using arbitrary NFEs.

Strengths

- The proposed method is well justified theoretically and the experiments validate the proposed approach. - The paper clearly states the connections between the proposed method and related existing solvers. - The experimental results include both pixel-space and latent-space diffusion and show consistent improvements when applying the method on top of existing solvers. - The method performance is superior to the baselines in both low and middle NFE regimes.

Weaknesses

1) It would be nice if the paper included the statistical significance of the FID scores reported in Tables 1-6. In the ~10 NFE regime sometimes these are very close to the baseline, while for lower quality low NFE one could expect the FID to be not very informative. I plan to revisit my decision after considering the statitstical significance of these results. 2) The paper repeatedly claims being able to achive higher order p than existing solvers. However I wonder how important it is in practice to achieve order p>3. Table 4 shows when including higher orders the results become poorer. And also I'd be curious to see this study for the higher quality NFE regimes (eg NFE=10). 3) Minor: - L. 39 "output of the model output" - L. 68-69 extra subscript 0 on lhs? - L. 71 "obtained" - L. 97 (after period) Can this be rephrased in a less subjective manner? - (5) I think $\hat{\epsilon}_\theta^{(k)}$ is not defined. Shouldn't the argument in this tem vary with $k$?

Questions

See weaknesses.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Yes.

Area Chair t2zM2023-08-18

Thanks for providing responses. All your comments will be taken into consideration when the committee makes decisions.

Reviewer uABf2023-08-18

Thank you for your rebuttal and the stastistical significance test.

The authors have satisfactorily addressed my concerns in their rebuttal.

Authorsrebuttal2023-08-22

Thanks for the response

Thanks a lot for the response. We are glad to hear that our rebuttal has satisfactorily addressed your concerns. Would you please consider raising your score?

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC