Taming Generative Diffusion Prior for Universal Blind Image Restoration

Summary

This paper proposes a blind image restoration method by using a pre-trained diffusion model without additional prior knowledge. The proposed adaptive guidance scale is fancy which uses the loss function to judge its value while the degradation function design is confusing. The results on real-world benchmarks show great performance success, but the experiment to validate their proposed modules is fragile.

Strengths

1. The author proposes a non-training strategy to handle the blind image restoration tasks with no additional prior knowledge and achieve SOTA performance in the real world. 2. The way to control the guidance scale is fancy.

Weaknesses

1. The introduction of the method section is confusing, first, what is the “$\sum$” in Line 112 and Line 421? Second, the design of the degradation function D does not make sense, why does adding the M term estimate the noises and what does the noise mean here (Line 72)? 2. The proposed method is unreliable, the author said that they do not have additional prior information, but in my view, the usage of the pre-trained model (Diff-BIR) is the prior knowledge, this paper is just an extra refiner to refine a coarse-clean image to a better one. To validate the proposed method, do the extra ablation study on all the benchmarks without pre-trained model and show the quantitative and qualitative results w, w/o it. 3. The ablation study is fragile. i) Visualize the result of the degradation function D for degrading the clean image. ii) Visualize what M learned for different tasks. iii) Show the value changing of guidance scales during the denoising stage and propose the theoretical analysis of the changing trend.

Questions

See the weakness.

Rating

3

Confidence

5

Soundness

2

Presentation

2

Contribution

2

Limitations

Yes.

Summary

This research introduces BIR-D, a novel approach to the universal challenge of blind image restoration. It leverages an adaptable convolutional kernel designed to emulate the degradation model, with the capability to refine its parameters progressively during the diffusion process. Furthermore, the work presents an empirical formula to guide the selection of the adaptive scale, a critical component in enhancing restoration accuracy. Extensive experiments substantiate the method's exceptional performance across a spectrum of restoration tasks, showcasing its robustness and efficacy.

Strengths

This research offers a novel perspective on Classifier-Guidance, highlighting the essential role of the guidance scale in the fidelity of image generation, and points out that applying a fixed guidance scale across all denoising steps is far from ideal. Therefore, it is necessary to innovate a method that enables the adaptive, real-time adjustment of the guidance scale at each stage of the diffusion process for degraded images in specific restoration tasks. The paper presents a robust validation of the BIR-D method through a comprehensive set of experiments across multiple image restoration tasks, such as deblurring, super-resolution enhancement, low light image enhancement, HDR image recovery, and multi-degradation image restoration.

Weaknesses

1. The contribution in question, which utilizes an optimizable convolutional kernel to simulate the degradation model and dynamically update the parameters of the kernel during the diffusion steps, may be perceived as lacking in novelty. You should provide a detailed comparison with the referenced [10], "Generative Diffusion Prior for Unified Image Restoration and Enhancement," about the different strategy for updating the degradation model. 2. The paper's exploration of the 'optimizable convolutional kernel' and 'adaptive guidance scale' could be enhanced by including an analysis of convergence trends or parameter behavior over time. Such analyses would clarify how these elements contribute to the method's performance. 3. The notations in this paper may lead to misunderstandings. Specifically, in Formula (6), the representation of $g$ lacks the subscript $xt=μ$, which is critical for clarity. Furthermore, the $N$ in Formula (19) should be distinguished from the $N$ used in Formula (20) to avoid ambiguity. Additionally, on line 418, the symbol $K$ should be replaced with $N$ for consistency. 4. The paper's explanation is not sufficiently clear, such as how BIR-D can accomplish multi-guidance blind image restoration.

Questions

1. Upon my review, the formula for calculating the guidance scale $s$ in equation (3), once combined and simplified with equation (1), yields an identity. This suggests that the information obtained from the current $xt$ sampling is independent of the update to $s$. Could you clarify this issue?

Rating

5

Confidence

4

Soundness

2

Presentation

3

Contribution

3

Limitations

None

Summary

The paper introduces BIR-D, a novel approach utilizing generative diffusion models for blind image restoration without requiring predefined degradation types. Traditional methods assume degradation models and optimize their parameters, limiting their applicability. BIR-D overcomes this by employing an optimizable convolutional kernel that simulates degradation dynamically during diffusion steps, allowing it to handle various complex degradations.

Strengths

The method stands out by integrating an optimizable convolutional kernel to dynamically adapt the degradation model during the diffusion steps, a concept not previously explored in the literature. The introduction of an empirical formula for adaptive guidance scale is innovative, eliminating the need for manual grid searches and enhancing the practicality of the approach across diverse image restoration tasks. The experimental results are robust, covering both qualitative and quantitative analyses on real-world and synthetic datasets. The superiority of BIR-D over existing methods is clearly demonstrated through comprehensive experimentation.

Weaknesses

The reviewer appreciates the innovative use of an optimizable convolutional kernel to dynamically adapt the degradation model during the diffusion steps. This is considered the most significant contribution of the work. However, this section lacks sufficient analysis and visualization. While the paper asserts that GDP [1] assumes specific degradation types and is not suitable for complex degradation models, the differences and improvements of the proposed degradation model compared to the one in GDP are not clearly explained. Additionally, the paper is missing some relevant references for blind IR [2] and earlier generative prior-based IR methods [3,4]. Overall, the reviewer appreciates the work and would be happy to adjust the rating if the aforementioned concerns are addressed. [1] Generative diffusion prior for unified image restoration and enhancement. CVPR'23 [2] AND: Adversarial neural degradation for learning blind image super-resolution. NeurIPS'23 [3] Image restoration with deep generative models. ICASSP'18 [4] Maximum a posteriori on a submanifold: a general image restoration method with gan. IJCNN'20

Questions

See weaknesses.

Rating

7

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors have addressed the limitations.