Diffusion Priors for Variational Likelihood Estimation and Image Denoising

Summary

This paper proposed a diffusion-based image denoising method. The method leverages the MAP framework with proposed adaptive likelihood estimation method and a pre-trained diffusion prior. Experiments on four real-world datasets validate the advantages of the proposed method.

Strengths

1. The paper is well-written and easy to understand. 2. The integration of a reverse diffusion process with adaptive likelihood estimation is novel. 3. The denoising performance of the proposed method outperforms that of other single-image denoising methods. The experimental results are substantial.

Weaknesses

1. The title of the paper does not accurately reflect its contributions. The use of diffusion priors to address inverse problems is widely studied, as the author has noted, and this paper builds upon the framework presented in [1]. The significant contribution of this paper appears to be the proposed alternative updating scheme for estimating the likelihood term. However, the title is too generic and does not capture this contribution. 2. The denoising computational cost is high, and the running time for the reverse diffusion process is lengthy. 3. In Algorithm 1: Difusion -> Diffusion.

Questions

The proposed algorithm involves numerous hyperparameters. How are these selected, and is the algorithm robust to variations in hyperparameter settings?

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors have acknowledged the limitations of their work.

Summary

The authors propose a way to use diffusion priors for real-world image denoising where the noise statistics are complex and signal-dependent. They use variational inference to estimate the joint posterior of the noise precision and image throughout diffusion time. The result is a MAP estimate of the denoised image that takes into account signal dependence and spatial correlation of the observed noise. The authors also propose to use a low-resolution diffusion model to denoise high-resolution images. In experiments, the authors compare to single image-based denoising methods, a self-supervised denoising method, and diffusion-based methods. They show improved quantitative and qualitative performance against the baselines.

Strengths

* The proposed method is sound and addresses a difficult problem. * Plenty of baselines are provided, and experiments show convincing quantitative and qualitative performance. Plus a good number of ablation studies is provided. * The proposed method works for other non-Gaussian synthetic noise as well as real-world camera noise.

Weaknesses

* The proposed method only provides MAP estimates (not posterior samples). * Nit: Eq. 6 should be explained more clearly in the text. For example, A should be explicitly defined. And why is the same $\epsilon$ used twice? Shouldn’t it be $y_{t-1}=\sqrt{\bar{\alpha}_{t-1}}(x_0+A\epsilon_1)+\sqrt{1-\bar{\alpha}_{t-1}}\epsilon_2$ for $\epsilon_1,\epsilon_2$ independently drawn (since the measured noise is different from the synthetic noise added in the diffusion process)?

Questions

* How difficult would it be to adapt this method to provide posterior samples?

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

I appreciate the authors providing a limitations section. There are probably more limitations of the method that could be discussed, such as not addressing posterior sampling.

Summary

This work considers the problem of using adapt diffusion models for solving real-world image denoising problem, that is, the noise is not assumed to be i.i.d Gaussian. The authors statistically model the real-world noise as independent, non-identically distribution noise, and then incorporate the adaptive MAP inference into the reverse process of the diffusion model.

Strengths

N/A

Weaknesses

1. The paper does not well motivate their choice of i.ni.d noise model. It is not clear why such an i.ni.d noise model can properly characterize the statistics of the real-world noise considered in the paper. 2. The paper is not well-written, lacking proper motivation and explanation of the design of the proposed method. 3. The approximation adopted in Eq.15 may lead to incorrect $x^\ast$. For example, when the function is non-convex, maximizing its Jenson low bounder produce an estimate that is far away from $x^\ast$.

Questions

1. Why is an i.ni.d noise model used to characterize the real-world noise? 2. Why do we want to adaptively update the parameters in reverse diffusion? Please provide some high-level explanation. 3. Please justify the accuracy of the approximation used in Eq. 15. 4. In Fig.3, self2self shows the best numerical performance. Please discuss the result.

Rating

5

Confidence

3

Soundness

2

Presentation

2

Contribution

2

Limitations

The authors have discussed the limitations. --------------------- After Rebuttal -------------------- The responses have addressed my questions. I have raised my score to 5.

Summary

Overall, this paper presents a novel method in real-world image denoising by proposing a sophisticated method that combines adaptive likelihood estimation, MAP inference, and variational Bayes within the diffusion model framework.

Strengths

1. Utilizing variational Bayes to dynamically infer the precision posterior is innovative and provides a more accurate modeling of real-world noise. 2. By exploring local priors in low-resolution diffusion models, the method can directly handle high-resolution noisy images without extensive patch-based or resize-based operations. 3. The proposed method's effectiveness is demonstrated through extensive experiments on diverse real-world datasets, showcasing superior performance compared to other unsupervised denoising methods.

Weaknesses

1. The method relies on hyperparameters (e.g., prior precision, temperature parameter, kernel scale etc.), which might require careful tuning for optimal performance for different datasets. Authors criticize previous methods for being dependent on hyperparameters but do not overcome such limitation in this paper. 2. The contribution of local Gaussian convolution might be overclaimed. As shown in Table 3, the performance improvements brought by rectification are pretty limited, which look like experimental variations.

Questions

1. What is the NFE for DDRM used in this paper? What is the inference time for DDRM and your method respectively? 2. In Algorithm 1, the threshold is set to 1e-6. Is performance sensitive to different values?

Rating

6

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

Authors discussed the limitation in the paper.

Summary

The paper proposes a method to tackle real world noise using an adaptive likelihood estimation . For this the authors develop a technique to dynamically infer the precision posterior using variational bayes. The authors perform comprehensive evaluation on real world denoising datasets to show the effectiveness of the method.

Strengths

1. The authors propose an method that could adaptively estimate the posterior distribution and make efficient use of off the shelf methods for real world denoising 2. The authors find that local priors from diffusion models pre-trained with LR image are more effective in real world denoising tasks. 3. The paper is well written and easy to follow. Derivations seems to be accurate.

Weaknesses

1. The authors have utilized a gamma based hyperprior without reasoning the design choice or comparing with other possible prior distribution. In order to show the need for a gamma prior, further experimental analysis are required 2. The methods seems to be applicable only for real world denoising limiting its practical utility. 3. Analysis with key diffusion based restoration methods like [1,2] are missing [1] Diffusion Posterior Sampling for General Noisy Inverse Problems [2] Manifold Preserving Guided Diffusion 4. [2] achieves good results with just 50 steps of sampling.Could the authors provide an analysis of the variation of performance with different number of sampling steps.

Questions

1. Can the authors provide an intuition of utilizing a gamma hyperprior over a gaussian prior like in [1,2]. [1] Diffusion Posterior Sampling for General Noisy Inverse Problems [2] Manifold Preserving Guided Diffusion 2. Is the method generalizable to other linear inverse problems than denoising [1,2] and DDRM, are all generic methods providing a general formulation to all linear inverse problems. 3. Could the authors provide an analysis of the variation of performance with different number of sampling steps.

Rating

5

Confidence

4

Soundness

2

Presentation

4

Contribution

2

Limitations

Yes, The authors have presented the limitations in a separate section.

Thanks for the response. Score raised.

I thank the authors for the thorough response to my comments. I found them very clear and helpful for me to better understand the motivation of the noise model and derivation of the proposed method. After a careful re-evaluation, I have raised my score from 3 to 5.

Thanks for the response.

Thank you for your response. Regarding using fewer sampling steps, are you referring to the DDIM sampler? Additionally, utilizing advanced diffusion samplers, such as DPM-Solver, may help accelerate the sampling speed. I have no further concerns and will maintain my score.