General Response
We appreciate the valuable feedback from all the reviewers. We are glad that reviewers found our idea interesting (`qJGK`), our framework efficient (`dyyk`) and simple yet effective (`jNDi`), and the paper well-written (`jNDi`) with Good use of diagrams (`qJGK`).
In this response, we provide:
1. additional inference *results on real-world images* (`qJGK`,`dyyk`);
2. new experiments on new concept axes extension *without retraining existing encoders* (`dyyk`);
3. additional quantitative evaluation results (`qJGK`,`jNDi`);
4. more qualitative results for baseline comparisons (`jNDi`);
5. further clarifications.
We include visual results for rebuttal in the `Rebuttal PDF` which is uploaded through the “Supplementary Material” button for easy reference [here](https://openreview.net/attachment?id=juuyW8B8ig&name=supplementary_material). In the responses below, we refer to figures and tables in this `Rebuttal PDF`, unless otherwise specified.
All new materials and changes have also been updated in the full manuscript and appendix marked in blue [here](https://openreview.net/pdf?id=juuyW8B8ig).
**Q1 - Inference Results on Real-world images (`qJGK`,`dyyk`)**
We provide new inference results on real-world images in Figures 1 to 5 in the `Rebuttal PDF`, spanning various types of objects, including furniture, kitchen utensils, fruits, and artwork. Despite *only* being trained on images generated by diffusion-based models, the concept encoders generalize well to diverse, complex real-world images, including *unseen* types of objects, materials, colors, and styles. Note that some of the examples originally presented in the main paper also came from real photos, such as Figure 5 from the main paper.
**Q2 - Extending with new concept axes (`dyyk`)**
Our model can be extended with more concept axes by training new concept encoders *without retraining the existing ones*, as shown in Figures 9 and 10. Given a model trained with concept encoders for `category` and `color`, we additionally train a third encoder for `material` while keeping the other two encoders *frozen*. With this progressive training procedure, the model is able to extend to new concept axes while maintaining the disentanglement of the frozen encoders.
**Q3 - Additional quantitative evaluation (`qJGK`,`jNDi`)**
To better quantify the performance on capturing nuanced colors, we designed a new metric based on color statistics for the task of concept editing as an additional quantitative evaluation.
Specifically, we compare the color of a 16 × 16 patch cropped around the center (focusing on the object) from both the input image and the image generated by the method being evaluated. The original size of the image is 64 × 64. We then compute the MSE of the mean color of the two patches, and report the average of the metric across all examples in Figure 11 in Table 1. Results suggest that our method captures colors more accurately compared to the baselines.
**Q4 - Additional visual results corresponding to the CLIP-Score evaluation (`jNDi`)**
In addition to the results in the main paper, in Figure 11 of the `Rebuttal PDF`, we provide more qualitative examples accompanying the experiments in Table 1. From these visual examples, we observed that the color nuances captured by ours are more accurate compared to the BLIP-2 baseline. However, since the CLIP-based metric specified in Section 4.3 of the paper cannot fully capture the minute differences, the BLIP-2 baseline still achieves comparable scores to our method despite this evident gap in visual results. To better quantify such visual differences in colors, we further designed a new metric based on color statistics, as explained in Q3 above.
**Q5 - Include BLIP-based baseline results also in Table 1 (`qJGK`)**
Thanks for the suggestion. We have included the numbers for the BLIP-based baseline in Table 1. In particular, we reran the human study with 20 users including this baseline in the comparison, and updated the results in the Table (copied below), which show significantly better editing quality. Note that since the human study measures the relative performance of all baselines, the scores for all methods are updated compared to the version in the original paper.
| Method | Edit Category | Edit Color |
|:--------------------|:--------------:|:-----------:|
| Null-text Inversion | 0.287 | 0.316 |
| InstructPix2Pix | 0.233 | 0.648 |
| Baseline w/ BLIP-2 | 0.448 | 0.379 |
| Ours | **0.968** |**0.840** |
**Q6 - Interpolation results**
We further demonstrate concept interpolation results. By interpolating between two concept embeddings, our model can generate meaningful images depicting gradual changes from one concept to another, such as meaningful hybrids of cherries and bananas shown in Figures 6 and 7. The details of the interpolation procedure are specified in Section 2 of the `Rebuttal PDF`.