Interpreting and Analysing CLIP's Zero-Shot Image Classification via Mutual Knowledge

Contrastive Language-Image Pretraining (CLIP) performs zero-shot image classification by mapping images and textual class representation into a shared embedding space, then retrieving the class closest to the image. This work provides a new approach for interpreting CLIP models for image classification from the lens of mutual knowledge between the two modalities. Specifically, we ask: what concepts do both vision and language CLIP encoders learn in common that influence the joint embedding space, causing points to be closer or further apart? We answer this question via an approach of textual concept-based explanations, showing their effectiveness, and perform an analysis encompassing a pool of 13 CLIP models varying in architecture, size and pretraining datasets. We explore those different aspects in relation to mutual knowledge, and analyze zero-shot predictions. Our approach demonstrates an effective and human-friendly way of understanding zero-shot classification decisions with CLIP.

Paper

References (64)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer wbb96/10 · confidence 4/52024-07-08

Summary

The paper investigates Contrastive Language-Image Pretraining (CLIP) for zero-shot image classification by exploring the mutual knowledge between visual and textual modalities. The study examines which concepts are commonly learned by both CLIP encoders and how they influence the shared embedding space. Using textual concept-based explanations, the researchers analyzed 13 different CLIP models, varying in architecture, size, and pretraining datasets. The approach provides insights into zero-shot predictions.

Strengths

- Introducing mutual information is reasonable. - The method is simple and effective. - The authors provide comprehensive experiments and analysis.

Weaknesses

- The introduction or Figure 1 does not clearly explain the motivation of the paper, making it difficult for me to understand. - There is extensive literature on the generalizability (zero-shot) of CLIP, and this paper lacks comparison and discussion with the latest works.

Questions

- What impact does the method proposed in this paper have on other fields (e.g., domain generalization)? - Figure 3 and Figure 5 are not clear enough. - Table 3 is not centered-aligned. - Some tables in the appendix are missing top borders.

Rating

6

Confidence

4

Soundness

3

Presentation

2

Contribution

2

Limitations

See Weaknesses and Questions.

Reviewer Whov5/10 · confidence 3/52024-07-10

Summary

The work deals with explainable artificial intelligence (XAI) in a multimodal (text-image) context. The proposed approach consists of first identifying the most important patches of a collection of images, and associating them to a textual description through CLIP. Hence, relying on a textual description of both the images and some textual description of the classes, the approach computes their mutual information for a varying number of visual concepts (still described by text), resulting in a curve from which it derives a score (AuC) that represent the "mutual information dynamics". The approach is evaluated on Imagenet for the task of zero-shot classification.

Strengths

* Analyzing explicability through the prism of mutual information between visual and textual features is original and potentially fruitful. The proposed approach consists to project the visual part to textual one but one could imagine further works * The paper reports a quantitative estimation of the explainibility, using four metrics previously defined in [7, 41]. The proposed approach exhibits good performance in comparison to three recent methods, some having been adapted to the multimodal case. * The method is tested with several visual encoders, from several types (visual transformer and several convolutional neural networks).

Weaknesses

* the presentation of the method in Section 3 is quite hard to follow: - first, it starts by "CLIP [65] formulates image classification as a retrieval task by using the textual class labels (...) and encoding them with the language encoder." which is a very imprecise way of putting things. CLIP does not "formulate" any task, it is a neural model that learns visual representation with natural language supervision. It can be instructed in natural language to perform image classification but it requires a specific approach (textual prompt) in each case. - the notations on lines 107-114 may be clearer. For example, it starts by considering an image $I$, then describes CLIP by splitting it between an encoder and a final linear layer that are no longer used in the following, to finally considering the similarity $s(i,j)$ between an image (that was $I$ at the beginning) and a text (never introduced) - lines 115-121, the text is unclear. The sentence "a unified set of $\mathcal{D}$ class-agnostic textual descriptors of length D" seems to mean that there are $\mathcal{D}$ 'descriptors' and that each is of length D. However, the following leads to doubt about that since the manuscript reports "D = 4,229 after discarding repetitive descriptors across the entire pool" suggesting that D is the number of 'descriptors'. Moreover, on line 118 "Concepts in $\mathcal{D}$" suggests that $\mathcal{D}$ is now the name of the set and not the number of concepts. After several readings, I think that the authors refer to the "length of the set" as Python programmer while one should consider the *size* of the set, meaning that D=4,229 is the number of elements in the set $\mathcal{D}$. - it is nevertheless still unclear what is a 'descriptor'. On line 120, an example of 'textual descriptor' is given ("can be hung from a tree") suggesting that it is actually a *description*. However, it is unclear how these descriptions are obtained, the only hint being on line 115: "We utilize an LLM to generate descriptors...". How these LLMs are used? From which data (what is the prompt)? While lines 102-114 introduce many notations just before, the names of the classes (one guesses $\mathcal{Y}$ on line 103) are no longer used in the following - on line 134, it is not clear what is "the zero-shot predicted class" nor how it is obtained. After some consideration on mutual information (lines 122-126) the overview of the method starts with a "given set of images" (line 127) thus it is unclear to which image (images? patches?) this "zero-shot predicted class" relates to. - In the same vein, the (same?) "zero-shot prediction of CLIP" at the beginning of section 3.2 should relate to an image but it is not clear which one - Line 205, how the "importance to the image" is determined? - Last but not least, it may be worth explaining how the "explanation" is provided by the proposed model once the "mutual information dynamics" is obtained * Similarly, Section 4 starts by explaining that an evaluation was conducted but the reader does not know for which task nor on which dataset; ImageNet is used for training (line 222) thus one guesses that it deals with zero-shot image classification (cf. title of the article) on the validation set of IM-1000? However, no implementation is provided for the experiment which results are reported in Table 1: which visual encoder is used (several are considered in the following)? how are they trained? Are the Vit/CNN pre-traind on some data? Which textual encoder is used? what is the LLM used to get the textual description of the classes and how is it done in practice? Moreover: - the manuscript refers to [7] to explain the metrics "Accuracy Drop" and "Accuracy Increase" (lines 224-225). However, in [7] the word "Increase" does not appear and the only occurrence of the word "Drop" does not refer to a metric. It is only in Appendix F that the reader can understand that the name of the metrics was changed. Moreover, if the "adaptation" of the metric is explained (still in the appendix) for the proposed method, it is not clear whether such an adaptation is required for the other methods (and whether it is the same as for the proposed approach). Previous works such as [37, 38] report the accuracy of each model, which seems relevant. * Previous works such as [18, 37, 38] usually evaluate their approach on several datasets. However, one must admit that they do not report quantitative evaluation in terms of explainibility. * On the opposite, it would have been relevant to compare the approach to those that specifically deal with XAI such as [a,b,c], in the vein of [d] which is a reference that could be cited. More generally the proposed work dealing with prototypes, it could be relevant to cite some of the works that adopted such scheme since [e]. [a] Wang, C., Liu, Y., Chen, Y., Liu, F., Tian, Y., McCarthy, D. J., Frazer, H., and Carneiro, G. (2023). Learning support and trivial prototypes for interpretable image classification. ICCV [b] Lei, Y., Li, Z., Li, Y., Zhang, J., and Shan, H. (2023). Lico: Explainable models with language image consistency. NeurIPS Information Processing Systems [c] Wan et al (2024) Interpretable Object Recognition by Semantic Prototype Analysis, WACV [d] Chen et al (2019) This Looks Like That: Deep Learning for Interpretable Image Recognition. NeurIPS [e] Saumya Jetley, Bernardino Romera-Paredes, Sadeep Jayasumana, and Philip Torr (2015) Prototypical Priors: From Improving Classification to Zero-Shot Learning. BMVC **minor points** - the text in Figure 1 is barely legible. On a standard A4 page, it is hard to read the list of concepts above (d). - reference [21,52] are incomplete - reference [29] is wrong, it has been published in NIPS 2012. Actually, a 2017 paper was published in Communication of the ACM, but it is clearly written (bottom right) that the original paper was published at NIPS 2012. - the date is missing in [30] - [49] both refer to the article and the (arxiv) preprint - line 224: low --> lower - The derivation in Appendix C is not very useful as the result is well-known for uniform distributions. However, a discussion on such uniformity may nevertheless be relevant.

Questions

Globally, many points are unclear as explained in the "weakness" section. Several implementation details should be reported in the manuscript, in particular which textual encoder is used? what is the LLM used to get the textual description of the classes and how is it done in practice? As well, how the "explanation" is provided by the proposed model once the "mutual information dynamics" is obtained?

Rating

5

Confidence

3

Soundness

2

Presentation

1

Contribution

2

Limitations

Two limitations are discussed in the Appendix (section D), one relating to the visual concepts that are not located in an image and the other to the type of tasks addressed (classification only). It nevertheless ignores the limitations linked to the usage of an external LLM to create the description of the classes, the limited evaluation (on one dataset only) not to mention the influence of bias in data in practical cases (that is an important issue to consider for the works dealing with explanations)

Reviewer BMwb6/10 · confidence 4/52024-07-19

Summary

The authors propose to interpret CLIP models for image classification from the lens of mutual knowledge between the image and text encoders of CLIP. Specifically, the authors use textual concepts as the common medium of the two modalities by mapping visual concepts to textual concepts. The authors then calculate mutual information in the shared textual space for analysis. Extensive experiments on a set of CLIP models demonstrate the effectiveness of the proposed method in understanding CLIP’s zero-shot classification decisions.

Strengths

- The proposed method is well motivated. The idea of mapping visual concepts to textual concepts and using the mutual concepts in the shared textual space for interpretation is interesting. - The paper is generally well-written and easy to follow. - The authors provide a comprehensive analysis on a broad set of CLIP models varying in architecture, size, and pre-training data. The experiments are extensive and the results are promising.

Weaknesses

- In L115, the authors use LLM to generate descriptors for all classes. However, it seems that some details are missing here. How exactly the descriptors are generated? What specific prompts and LLM are used? Is there any filtering mechanism to remove noisy text descriptions? How to maintain the relevance and diversity of the generated descriptors for each class? It seems that the design of prompts tends to affect the generated concepts a lot. The authors should also provide ablations on the effect of generated concepts as well. - Given the noisy nature of PCA/K-means used for visual concepts, how do the authors filter these failure cases? - In Table 1, the proposed method does not perform well on the Insertion Metric. Could the authors provide the justification for this?

Questions

See the questions mentioned above. I am leaning towards borderline accept and hope the authors could address my concerns during the rebuttal.

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

The authors have addressed the limitations in Sec. D (supplementary material), which looks good to me. It would be better to discuss some broader societal impacts of the work as well.

Authorsrebuttal2024-08-11

Dear Reviewers, Thank you again for your time and effort in reviewing our paper. As the discussion period is coming to an end, we would kindly like to ask you to go through our rebuttal. We believe that we have sufficiently addressed all your concerns and sincerely hope for reviewers BMwb and Whov to raise their scores. Thank you and good day!

Reviewer BMwb2024-08-12

Official Comment by Reviewer BMwb

Thanks for the authors' response. The rebuttal well addressed my concerns. I would like to increase my score to 6. I suggest the authors incorporate all the discussions into the final version.

Reviewer wbb92024-08-12

Official Comment by Reviewer wbb9

Thanks for the authors' response. I kept the original score.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC