Summary
This article presents a tuning-free inpainting method based on stable diffusion. The authors propose an attention-guided strategy and a classifier optimization algorithm, achieving promising results in text-based inpainting.
Strengths
1. The method is distinguished by its training-free nature, resulting in heightened computational efficiency.
2. It demonstrates the capacity to harness the inherent image priors of the diffusion model, thereby yielding favorable outcomes, particularly in the context of object removal.
3. The article exhibits a well-structured presentation with lucid exposition and a straightforward implementation process.
Weaknesses
1. The attention strategy proposed in the article entails multiple hyperparameters. While the authors provide default values, empirical evidence suggests that achieving satisfactory results often necessitates multiple manual adjustments for each image, rendering the process notably inconvenient.
2. The use of attention-based adjustment strategies is a well-established practice in numerous image and video editing tasks, as exemplified by applications such as P2P and FateZero. While the authors have made some adaptations for inpainting tasks, the overarching concept remains largely derivative, lacking substantial innovation.
3. The experimental methodology employed in the study exhibits several shortcomings. The comparison is limited to the ImageNet dataset, overlooking widely used inpainting datasets like FFHQ, CelebA, and Places. Furthermore, the authors rely solely on FID as an evaluation metric, neglecting common indicators such as U-IDS, which compromises the persuasiveness of their findings. Notably, the comparative analysis of text-based inpainting methods is limited, with only one alternative considered, thereby omitting important benchmarks such as NUWA-Inpainting.
4. The ablation study conducted by the authors consists of a single figure, offering limited persuasive value. A more compelling approach would involve a more comprehensive array of case studies and quantitative comparisons.
5. A substantial challenge associated with attention-based methods is their potential to compromise fidelity. This issue is also evident in the method under examination, as evidenced by Figure 9 and Figure 11, where numerous images exhibit noticeable alterations in color and tone.
6. The inference efficiency is not reported. Comparisons to GAN-based and DM-based methods are required.
7. The proposed method is object-oriented. It would be better to provide more cases for background completion.
Questions
1. Is the setting of lambda, t1, and t2 related to the size and shape of objects?
2. Can the proposed method effectively handle situations with occlusion?
Rating
5: marginally below the acceptance threshold
Confidence
5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.