Q1: Upsampling LR images to match the target pixel size is a commonly used technique in diffusion-based SR methods. While this technology is not new, our main contribution lies in combining upsampling with noise injection to utilize diffusion model for ASSR tasks. We have also conducted a mathematical analysis to explain why this approach can be effective. We acknowledge that our previous wording may have caused some confusion as Reviewer [c56f] said and we will be more careful with our choice of words and expressions in our revised version.
| Model | Scale | DIV2K-FID | DIV2K-PSNR | DIV2K-SSIM |
| ---- | ---- | ---- | ---- | ---- |
| StableSR | 4x-downsample | 1.962 | 29.615 | 0.688 |
| StableSR | 2x-downsample | 0.219 | 30.004 | 0.827 |
Q2: The concern you raised regarding computational cost is a potential issue that exists in all current diffusion-based research. Our method naturally reduces some computational overhead by injecting noise only to a portion of the original image. Instead of starting from pure noise to generate the image, we begin the denoising process from the middle point, saving computational resources. Additionally, we have employed the DDIM sampler to reduce the total number of sampling steps to 50. Furthermore, consistency models suggest that one-step denoising can directly yield the original image from any intermediate result. Although we consider this as a potential solution for computational overhead, we leave it as future work.
Q3: Our codebase is built upon the original DDPM [1]. In this codebase, the base-channel for the U-Net is set to 64. However, the latest DDPM codebases have larger base-channels, such as 256 (DDRM, DDNM) and 320 (LDM). This difference in parameter settings creates a gap between our model and other DGM models. Due to limitations in computing resources and time, it is currently challenging for us to train with a higher base-channel. We did attempt to reduce the base-channel from 64 to 32 to alleviate training overhead, but this led to a significant decrease in model performance.
Q4: DDNM and DDRM are two recent studies that we consider relevant to our work. We acknowledge that StableSR is a novel diffusion-based SR model, and we have included its results for comparison. However, it is important to note that the experimental comparison may not be entirely fair, as StableSR can only upscale LR images to 512, while our experimental settings require a target pixel size of 256x256. Therefore, we had to downsample the StableSR image to match our target size. We provide this result as a reference, understanding that the comparison may not be entirely equitable.
Q5: As your suggested, we followed the degradation pipeline designed by ESRGAN and tested it using the corresponding degradation code on DIV2K. We applied Blur, JPEG compression, and noise methods to degrade the original image. We then injected noise and denoised the image to evaluate the recovery effect of the model. Additionally, we used the two-order pipe from ESRGAN-pipe to degrade the image multiple times and finally reduced the image by 2 times. Interestingly, when using only a single type of degradation, the images were still repaired effectively. Among the mentioned degradations, the model performed best in recovering blur, but struggled the most with compression at 4x scale. Furthermore, we observed that the degradation results of image processing with a single degradation method were not visually obvious, especially in the case of JPEG compression. However, when applying ESRGAN's pipeline to degrade images multiple times, the model's performance significantly declined. Therefore, image recovery with multiple degradations remains a challenging task.
|Method| FID | PSNR | SSIM |
| ---- | ---- | ---- | ---- |
|4x| 1.088 | 32.450 | 0.924 |
|Blur| 0.331 | 33.583 | 0.953 |
|JPEG Compression | 0.127 | 32.295 | 0.954 |
|Noise| 0.104 | 32.220 | 0.947 |
|ESRGAN| 1.528 | 30.817 | 0.821 |
Q6: Yes, all of our experiments were conducted using the same experimental settings.
Q7: In our experiments, we used the same compression method for the same compression ratio. Therefore, we assumed that all the images had the same blur level. We also used the same parameters to inject noise and denoise the images during the enhancement process. However, when entering the formula (19) that we derived, we realized that the decision of noise level is actually related to the difference between the high-resolution (HR) and low-resolution (LR) images. If the initial blur level of the image is different, the image with a higher degree of blur will experience a greater fidelity loss, requiring more noise injection. From this perspective, the degree of blur does play an important role in the trade-off. Unfortunately, we were unable to find suitable mathematical tools to establish a connection between the Fidelity term and the degree of blur in this paper.
[1] https://github.com/lucidrains/denoising-diffusion-pytorch