Response (2/2)
>The concerns regarding the presentation are not resolved. Note that releasing the code does not help unless the code is also reviewed for completeness and clarity.
Could you provide further insight into the concerns surrounding the presentation? Within Section 4, we delineate our primary methodology. To delve deeper, Section 4.1 elucidates our approach to incorporating a novel concept into the model, which subsequently facilitates its use in text prompts. Subsequently, in Section 4.2, we expound upon the training-free attention swapping procedure. This comprehensive process is also visually represented in Algorithm 1. Should there remain any ambiguities, we are more than willing to offer additional clarifications. We deeply appreciate your feedback and suggestions.
>While I understand it is hard to "reproduce" the data collection process, "collecting from internet", "randomly sampled" are still to vague. Information regarding the data source, e.g. how they are crawled, the size, any selection process, etc. should be provided.
We acknowledge that our initial description of the source image collection process was not sufficiently clear. In this context, we provide an in-depth explanation.
For real images, we sourced all our images from internet searches. We employed the search prompt: 'a photo of <target>'. Here, the <target> variable could be a specific celebrity (e.g., 'Elon Musk') or a descriptive scene (e.g., 'a cute yellow cat running in the forest'). The celebrity names were identified through a Google search with the prompt "top celebrities 2023." For scene descriptions, we curated a list of 100 distinct search prompts to source images from the internet. In total, we aggregated 1,000 images using these prompts. All prompts, along with the collected image, will be made available in our next revision.
For synthetic images, we generated 1,000 images using text prompts with the text-to-image diffusion model version 2.1. These prompts spanned a range, including those centered on humans (e.g., "A photo of a woman looking left and smiling, Van Gogh style") and those focus on non-human subject (e.g., "An old car in the middle of the road, flanked by trees during autumn"). All prompts used in synthetic image generation will also be released too. For the human evaluation exhibited in this rebuttal and in the paper, we utilized the "random" package in Python to sample 200 images from both the real and synthetic datasets, respectively. Each image underwent evaluation by five distinct individuals on Amazon Turk. In total, this resulted in a comprehensive 6,000 ratings, as we compared our model against P2P, PnP, and MasaCtrl. Our findings unequivocally indicate that our model surpasses all other methods in performance.
For source image processing, all we do is to resize the image into standard 512x512 pixels. It is worth noting that there is also no postprocessing needed for generated images.
>It would be more informative if the authors can provide how the success rate is measured and what's the criteria for success. Also, a few qualitative examples in the paper are not sufficient to be considered a meaningful ablation study. To make it a meaningful study, the authors need to show how consistent the results are.
To the best of our knowledge, there is no existing automatic evaluation metric for success rate on image editing. Due to the lack of ground truth and standard evaluation metrics, current image editing works such as P2P[1], PnP[2], and MasaCtrl[3] are mostly using human evaluation and qualitative comparison to demonstrate performance.
In response to your feedback, we expanded our analysis to incorporate human evaluations focused on the success rate.
With respect to quantifying the success rate, we established three criteria to define a successful image swap:
* The generated image featuring the replaced subject should exhibit high quality.
* The background details not pertaining to the targeted subject should remain consistent with the original source image.
* The outcome of the swap should be recognizable as the subject from the concept image, mirroring the pose, gesture, and facial expression of the original subject.
Presented below are our findings, based on a sample of 200 real images juxtaposed with 200 synthetic images:
| | Ours | P2P+DreamBooth | PnP+DreamBooth | MasaCtrl+DreamBooth |
|-------------------|------|----------------|----------------|---------------------|
| Human subject | 0.57 | 0.37 | 0.34 | 0.12 |
| Non-human subject | 0.72 | 0.51 | 0.43 | 0.14 |
Table 6. Human evaluation results on Success Rate.
In the table, results from the human evaluator our method outperforms other methods in terms of success rate. Thanks for the insightful suggestions on the success rate, and we will include this in the next revision.