NoiseDiffusion: Correcting Noise for Image Interpolation with Diffusion Models beyond Spherical Linear Interpolation

Thank you for your valuable comments and constructive feedback on our manuscript. We appreciate the time and effort you have dedicated to reviewing our work. In response to your insightful suggestions, we have made several revisions to address the key concerns raised during the review process. **Q.1**: Propose a metric to quantify the performance on a large test dataset. > However, to further support this it would be useful to have some metric to summarize performance over a large test data-set.I appreciate this may not be trivial but I wonder if the authors have any thoughts on how they could assess performance over a large test-set? Could the authors propose a summary metric to qualtify the performance of their approach over a large test-data set? At present results are presented for just a handful of test images. While is seems very likely the improved performance of the proposed method will generalize to other data it would be more compelling if the authors could demonstrate this quantitatively. > ***Ans for Q.1***: Thank you for your constructive suggestions. Accordingly, we have conducted more experiments using difference models and datasets. These results are reported and discussed in our revision. - We have conducted experiments on Stable Diffusion model with datasets like eagle, lion-tiger, peach and so on. The high quality results are reported in the Appendix C.2, which shows that our method can consistently achieve good performance under different settings.Specifically, we have tested not only the scenarios where the drift coefficient of the model is zero and non-zero but also tested whether the model's variance explodes, which was done to determine whether the diffusion process belongs to a Variance Exploding SDE(VE-SDE) or a Variance Preserving SDE(VP-SDE)[1]. This indicates that our approach may be useful in almost all diffusion models. - We agree that an appropreate quantitative metric is crucial for compare difference methods. However, as you mentioned, designing such a metric is challenging since it should not only be based on pixels but also consider the semantic aspect to assess the quality of the interpolated images. The challenge may be the reason why previous methods in the literature did not involve such a metric. In this context, we would like to enhance the persuasiveness of our results by reporting additional results in the appendix. We will leave it as our future work to explore methods for quantifying the quality of interpolated images. **Q.2**: Boundary control are not explained. >Boundary control methods are introduce in Section 4.4 but are not explained. The appendix is referenced but that only contains an additional figure of results. The boundry control methods should be discussed somewhere. In general Appendix B feel incomplete. ***Ans for Q.2***: We apologize for missing the introduction of the boudary control. Following your valuable suggestion, we have highlighted the definition and explanation of boudary control in our revision of Chapter 4.5. Specifically,combining the "68–95–99.7 rule" and considering our analysis of how noise above the denoising threshold impacts images, data points exhibiting significant deviations from the mean are considered potential sources of image artifacts.Therefore, we control the boundary of noise. **Q.3**: Writing problem. >There seem to be a number of minor typos throughout where spaces a missing, typically following a period or comma. > ***Ans for Q.3***: Thanks for your kind suggestion. In respect to your careful reviewing, we have re-examined the entire paper and fixed all typos. Meanwhile, following your advice, we have added the citation of our theorem to the main text. [1]Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., and Poole, B. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021.

**Q.1**: Practical guidance for selecting these parameters remains unclear. > The paper's contribution heavily relies on specific parameter selection (e.g., parameters in eq.(14), boundary values in section 3.5, and the interpolation coefficient in slerp), yet practical guidance for selecting these parameters remains unclear. ***Ans for Q.1***: Thank you for pointing out the problem. - We agree with your point that the performance of our method is heavily related to the introduced hyperparameters. In this context, we would like to note that these hyperparameters are selected from a relatively stable range in our experiments. Specifically,$\gamma$ is within the range $[0, 0.1]$, $\mu$ and $\nu$ are within the range $[0.8, 1]$, the boundary is within the range $[2.0, 2.4]$. Users only need to set $\alpha$ and $\beta=\sqrt{1-\alpha^2-\gamma^2}$ can be automatically calculated. - We appologize for the missing strategy to select hyperparameters. In our experiments, we select hyperparameters from a subset of images (less than 20 images) and apply these hyperparameters to the remaining images. **Q.2**: Computational overhead and potential limitations. > Computational overhead and potential limitations of implementing the proposed method in practical scenarios lack detailed explanation. ***Ans for Q.2***: Thanks for your constructive suggestions. Accordingly, we have added the following explanation and discussion to our revision. - Our method involves an extra step compared to using SDEdit for interpolation, which is the mapping of the image to the latent variable. This extra overhead will double our processing time compared to interpolate images using SDEdit. As a result, this extra overhead leads to better feature preservation, as the noise and image information are quite balanced in this case. - Regarding potential application limitations, our paper mainly focus on image data. Thus, its effectiveness in other modalities has not been validated, which a potential limitation of our work. We will explore the posibility of our method on different modalities in our future work. Thanks again for your constructive suggestions, and we believe the above discussion could significantly improve the quality of our paper. **Q.3**: Further exploration to assess with diverse settings. >The experiments are conducted under a simplified scenario where the drift coefficient $\mu(x_t,t)$ is set to zero. Further exploration is needed to assess the level of improvement and applicability in diverse domains or image types, as seen in Figure 6. The paper lacks explicit discussion on the generalizability of the method to various image sets. ***Ans for Q.3***: Thank you very much for your valuable suggestions. Following your advice, we have conducted supplementary experiments and provide explanation to our revision. - We set $\mu(x_t,t)=0$ following the defaul setting of previous work [1], while overlooking the potential limitation of this experimental setting. Inspired by your valuable comments, we have conducted more experiments using Stable Diffusion, and reported the results in Appendix C.2. Specifically, Stable diffusion is built upon DDPM, and the drift coefficient of DDPM is non-zero. The results demonstrate that our approach performs quite well under various settings. - Follwing your valuble suggestion, we further verify the effectiveness of the proposed method on more datasets, including tiger-lion, peach and so on. The results are listed in Apendix C.2 demonstrating that our method can consistently achieve excellent results. Thanks again for your constrctive suggestions. We have added the above discussion to our revised paper. [1]Karras, T., Aittala, M., Aila, T., and Laine, S. Elucidating the design space of diffusion-based generative models. In Proc. NeurIPS, 2022.

Thanks a lot for your time in reviewing and reading our response and the revision. We sincerely understand you’re busy. But as the window for responsing and paper revision is closing, would you mind checking our response (a brief summary, and details) and confirm whether you have any further questions? We are looking forward to your reply.

We sincerely thank you for taking the time to review. According to your comments, we provide detailed feedback below and also add them into the revision. We believe that these changes could significantly improve the overall quality of our work: **Q.1**: It would be difficult to set the parameters. > First, the proposed method has a number of empirical parameters (alpha, beta, gamma, mu, nu, and boundary). It would be difficult to appropriately set those parameters in practice, although the effects of each parameter is experimentally demonstrated. ***Ans for Q.1***: Thanks for pointing out this potentially confusing problem. Accordingly, we have supplemented the relevant explanations in the revision. The introduced hyperparameters are listed as follows. Although our method introduces several hyperparameters, we would like to note that it is relatively effortless to tune the parameters. - i) Boundary determines the extent of noise constraint. In our experiments, the boundary is within the range $[2.0, 2.4]$. - ii) $\gamma$ determines the strength of the added noise. In our experiments, $\gamma$ is within the range $[0, 0.1]$. - iii) $\mu$ and $\nu$ supplement the information lost in the generation process (by introduing the information from the original image). In our experiments, $\mu$ and $\nu$ are within the range $[0.8, 1]$. In addition, our method involves a user-determined parameter $\alpha$. - $\alpha$ is set by users, since it controls the degree of interpolation bwteen two images, where larger $\alpha$ means the generated image is more similar to the first image. In this context, $\beta = \sqrt{1-\alpha^2-\gamma^2}$. **Q.2**: The experimental evaluation is limited. > Second, the experimental evaluation is limited: only qualitative evaluation on small number of images. ***Ans for Q2***: Thank you for your constructive comments. Accordingly, we have added the following experiments and explanations to our revisoin. - We have conducted experiments using more models and datasets. Specifically, we evaluate our method using Stable Diffusion model on eagle, tree, car and other datasets. The results are reported in the Appendix C.2. These reults show that our method can consistently achieve good performance under various settings. - We agree that an appropreate quantitative metric is crucial for comparing difference methods. However, designing such a metric is challenging in the literature, since it should not only be based on pixels but also consider the semantic aspect to assess the quality of the interpolated images. The challenge may be the reason why previous methods in the literature did not involve such a metric. In this context, we would like to enhance the persuasiveness of our results by reporting additional results in the appendix. We will leave it as our future work to explore methods for quantifying the quality of interpolated images. We believe these results and discussion would make our work more solid. Thanks again for your valuable comments. **Q.3**: The introduction and the references is too short. > Third, the presentation of this paper could be improved. In my opinion, Introduction is too short and the number of references are too small. ***Ans for Q.3***: Thanks for your kind suggestion. Accodingly, we have modified the paper to provide a detailed introduction and added more references. The corresponding content will be placed in the next response box.