Summary
This work introduces a new point-based image editing technique with high capability and effectiveness, DragDiffusion. The method can be applied to edit a variety of images for point-based editing, including real and synthesized images, as well as challenging cases such as images with multiple objects, surpassing the previous GAN-based editing method DragGAN both quantitatively and qualitatively. A new benchmark for point-based editing evaluation, DragBench, is introduced. It is a beneficial contribution to future research on image editing.
Strengths
- DragDiffusion is of high capability and can effectively accomplish point-based image editing tasks. In some cases, it can produce obviously better and more reasonable results than the DragGAN approach.
- The method is well-designed. Starting from Diffusion UNet LoRA fine-tuning, the method further leverages 1) the latent optimization technique to align latents with user editing, and 2) the latent-MasaCtrl strategy to ensure enough identity between the edited image and the input.
- DragBench is introduced for evaluating point-based image editing.
- Extensive experiments with abundant visual evaluations for illustrating the effectiveness of DragDiffusion. Ablation studies are conducted on the number of inversion steps and the number of LORA fine-tuning steps for revealing the influence of key parameters on the method.
Weaknesses
- The significance of the technical contribution is a little bit limited. Though with insights into the relationship between diffusion latent and the image content, the method relies on existing techniques to acquire good performance, including the latent optimization (from DragGAN) and the latent-MasaCtrl design based on the existing MasaCtrl strategy. Therefore, despite the effectiveness of the method and the smart usage of the insights on diffusion latents, it is hard to tell what new messages or general principles which are potentially useful for other works it can convey.
- The technique is proposed for point-based image editing. It would be better if the method (or with some small modification) could solve other image editing problems. The claim on its strong power, generality, and versatility would be more solid if it could be broadly applied to other editing problems.
- Showing more qualitative comparisons between DragDiffusion and previous or concurrent approaches such as DragGAN would be beneficial to evaluating its superiority in point-based image editing.
- Ablation studies are not particularly sufficient.
DragDiffusion is effective and superior to previous works. However, the main concerns lie in the significance of its technical contribution and the insufficient qualitative comparisons. Thus I vote the acceptance but limited to score 6.
Questions
More visual comparisons?
Other ablation studies?
Rating
6: marginally above the acceptance threshold
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.