Dear Reviewer oWB8
Dear Reviewer,
Thank you for your thoughtful feedback on our paper regarding composed image retrieval with text feedback. We appreciate your detailed analysis and constructive suggestions.
**Q1-1. Explain novelty. The difference from existing methods with convincing proof**
A: Thank you. Existing methods merely focus on the fine-grained search, by harnessing positive and negative pairs during training. This pair-based paradigm only considers the one-to-one distance between a pair of specific points, which is not aligned with the one-to-many coarse-grained retrieval process and compromises the recall rate.
In an attempt to fill this gap, we introduce an uncertainty learning approach to simultaneously modeling the coarse- and fine-grained retrieval by considering the multi-grained uncertainty.
It is worth noting that the proposed method is focused on better learning objectives, which is complementary to existing works on compositor design.
**Q1-2 / Q3. The baseline selection? More additional comparison methods and a more detailed discussion in Table 4(b). More SOTA methods.**
A: Thank you. Considering the availability of open-source code, we select a widely-used method, CosMo, as baseline. We also show the scalability of our method to a recent baseline, Clip4Cir in Table 4(b).
In Table 1, we have compared ARTEMIS and DCNet, both within the last three years.
We will further add FashionViL and Pic2Word in comparison as follows.
| Average on FashionIQ | R@10 | R@50 |
| -------------- | ----- | ----- |
| ARTEMIS [1] | 26.05 | 50.29 |
| DCNet [2] | 27.78 | 53.89 |
| Pic2Word [4] | 24.70 | 43.70 |
| FashionViL [3] | 26.60 | 52.74 |
| FashionViL* [3] | 31.21 | 57.04 |
| Ours | **33.17** | **61.39** |
(* Note that FashionViL introduces extra datasets for training. )
[1] Delmas et al. Artemis: Attention-based retrieval with text-explicit matching and implicit similarity. In ICLR, 2022.
[2] Kim et al. Dual Compositional Learning in Interactive Image Retrieval. In AAAI, 2021.
[3] Han et al. FashionViL: Fashion-Focused Vision-and-Language Representation Learning. In ECCV, 2022
[4] Saito et al. Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrieval. In CVPR, 2023
Besides, as suggested, we adopt a strong cross-modality baseline BLIP [5], and add our method. We observe that the proposed method could improve the performance as follows.
| Shoes | R@10 | R@50 |
| -------------------- | --------- | --------- |
| BLIP | 43.93 | 70.21 |
| BLIP + Ours | 48.40 | 75.29 |
[5] Li et al. BLIP: Bootstrapping Language-Image Pre-training. In ICML 2022
**Q2. Highlight the unique aspects in Method Section. Too simple?**
A: Thank you. One unique aspect is Eq. 5, which uniforms the fine-grained and coarse-grained learning. In particular, the fine-grained loss can be viewed as a special case of coarse-grained loss, whose variance is a constant term. In this way, we could unify the multi-grained learning losses and easily balance them via a dynamic weight $\gamma$. We also have a detailed discussion at the end of Method Section on page 6.
Yes. Simple is also our advantage. Despite its straightforward nature, our approach brings about significant performance improvements. On the three public datasets, i.e., FashionIQ, Fashion200k, and Shoes, the proposed method has achieved +4.03%, + 3.38%, and + 2.40% Recall@50 accuracy over a strong baseline, respectively.
This simplicity also ensures broader accessibility and applicability in various contexts. We hope that the simple method can serve as a new baseline and give the community a new perspective.
**Q4. The description of the first contribution.**
A: Thank you. In the first contribution, we hope to point out a long-exist misalignment problem, which compromises the training. The problem is due to the multi-grained annotation of the datasets. For instance, "change red" is a coarse-grained feedback, while "change red and short sleeves with a blue white Micky mouse on the clothes" is a fine-grained feedback. It is challenging to learn the multi-grained by the conventional one-to-one metric learning losses, like infoNCE loss, sphere loss and etc. We will modify the first point of motivation as follows:
*We pinpoint a training/test misalignment in real-world image retrieval with text feedback, specifically between fine-grained metric learning and the practical need for coarse-grained inference. Traditional metric learning mostly focuses on one-to-one alignment, adversely impacting one-to-many coarse-grained learning. This problem identification underscores the gap we address in our approach.*
**Q5. The standardization and uniformity of table**
A: Thanks a lot for your suggestion. We will re-arrange the table location and modify the table style to make its style more uniform.