LICO: Explainable Models with Language-Image Consistency

Interpreting the decisions of deep learning models has been actively studied since the explosion of deep neural networks. One of the most convincing interpretation approaches is salience-based visual interpretation, such as Grad-CAM, where the generation of attention maps depends merely on categorical labels. Although existing interpretation methods can provide explainable decision clues, they often yield partial correspondence between image and saliency maps due to the limited discriminative information from one-hot labels. This paper develops a Language-Image COnsistency model for explainable image classification, termed LICO, by correlating learnable linguistic prompts with corresponding visual features in a coarse-to-fine manner. Specifically, we first establish a coarse global manifold structure alignment by minimizing the distance between the distributions of image and language features. We then achieve fine-grained saliency maps by applying optimal transport (OT) theory to assign local feature maps with class-specific prompts. Extensive experimental results on eight benchmark datasets demonstrate that the proposed LICO achieves a significant improvement in generating more explainable attention maps in conjunction with existing interpretation methods such as Grad-CAM. Remarkably, LICO improves the classification performance of existing models without introducing any computational overhead during inference. Source code is made available at https://github.com/ymLeiFDU/LICO.

Paper

References (49)

Scroll for more · 37 remaining

Similar papers

Peer review

Reviewer r2cD6/10 · confidence 4/52023-06-29

Summary

This paper proposed LICO, which leverages the textual and semantic knowledge learned by large language models to guide latent image features. By matching relationships among images with KL-divergence globally, and distances between specific feature maps and prompt tokens with OT, the feature space of image is aligned with the prompt tokens in LLM.

Strengths

1. It provides a novel idea that uses the semantic knowledge of LLM to guide the feature representations in the image classification model. Since the feature expression ability of the model is improved, both the classification performance and interpretation maps generated by XAI methods are better. 2. Thorough experiments are conducted on multiple datasets and show the effectiveness of the proposed method. 3. The paper is clearly written and easy to follow.

Weaknesses

1. My main concern is that the model trained with the help of LLM is not the original model, so the improved interpretation is for the later model which has a better performance. So I'm unsure whether this can be regarded as "enhancing existing visual interpretation methods." since the interpreted model is already changed. It's more like LICO enhances the classification model so that the XAI results are improved. LICO is not an XAI method itself, but it helps the model better learn latent features by introducing knowledge from LLM. 2. Some small problems: - What’s the transport cost used in OT? - Since fixed promotes are in the original CLIP but learnable X1 to Xm-1 are added in the language model input, can the model work well with frozen parameters and without fine-tuning? How are the learnable prompts initialized? - Missing introduction in Sec.1 and Sec.2 for perturbation-based explanation methods like RISE, which is also compared in the experiments. - The legend for g and f in Figure 2 are reversed.

Questions

see the "Weaknesses"

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

3 good

Contribution

3 good

Limitations

N/A

Reviewer V7Pm5/10 · confidence 3/52023-07-01

Summary

This paper introduces LICO, a model that aligns visual encoder to language features. This model incorporates a frozen text encoder, a trainable image encoder, a classification loss, a manifold matching loss and an optimal transport loss. Experiments shows improvements over existing interpretation methods.

Strengths

1. This paper is well motivated, and the method is clearly stated in text as well as figures. 2. Quantitative results are better than baselines. 3. The experiments are comprehensive in terms of the number of datasets and baselines to compare to.

Weaknesses

1. LICO obtains class label text embeddings via a text encoder pretrained with internet-scale image-text pairs, thus bringing additional information. Further, the image encoder is trainable whilst in baseline methods the image encoder are all frozen. These two aspects add up to an unfair comparison to other baselines. 1. The optimal transform loss is motivated by aligning partial regions to prompt tokens, but there's no quantitative/qualitative experiments analyzing that specific effect. 2. The architectures used in this work are generally outdated, and exact version of the architecture (ResNet18/50) are less performant ones. 3. In Figure 3, the qualitative results from LICO are not better than baselines. 4. In Table 2, the improvements are not significant.

Questions

1. How would authors compare the manifold loss to a cross-entropy with softmax temperature > 1? 2. What is the necessity of the text encoder being from CLIP? Would a pure text embedding model (word2vec, bert, etc) work? 3. Would a model trained with LICO work for words that are not included in the training class label set?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

The authors discuss about their limitation on L303, which is not a true limitation compared to some in the Weaknesses I listed. Further, this limitation of "some training overhead" is not discussed in the paper. In concept, such an MLP would only bring insiginificant training cost when compared to the image encoder.

Reviewer rFY85/10 · confidence 4/52023-07-05

Summary

Most visualization interpretation methods based on saliency information often generate inaccurate saliency maps due to the limited discriminative information provided by one-hot labels. The manuscript proposes a language-image consistency model (LICO) to address this challenge. LICO utilizes a large-scale vision-language model CLIP to improve existing CAM-based vision explainable methods such as Grad-CAM. The authors assume that numerous image-text pairs used in CLIP can encode rich semantic information that can be aligned with the latent space of image domain, which establishes global manifold structure alignment and assigns local feature maps with class-specific prompts to generate more accurate saliency maps. The paper is well-organized and motivated, and the idea of leveraging language information from large VLMs is intuitive and effective. Experiments including deletion and insertion tests, sanity checks and classification demonstrate that LICO can achieve improvements over existing interpretation methods, resulting in more explainable attention maps.

Strengths

(1) The paper is well-organized and easy to understand. (2) The motivation behind leveraging language information from large VLMs is clear and effective. (3) The use of prompt information and multi-modal techniques on explainable methods may be with good value.

Weaknesses

(1) The effectiveness of LICO requires further verification. The sensitivity of LICO's saliency to model parameters raises concerns about its locality and uncertainty. Evaluations of more complex images and assessing the model's robustness would be beneficial. (2) The results of the saliency map are not sufficient to support the conclusion of better coverage of comprehensive and discriminative regions. Comparisons with more complex multi-target and multi-class images are preferable. (3) The details of the ablation studies are not clearly presented. The saliency maps generated by CAM-based methods + LICO may be insufficient to qualitatively show the salient regions that the classification model focuses on. It would be better to additionally provide results of “multi-objects single-class” and “multi-class”, except for “single-object”. Moreover, it would be better to conduct additional experiments to evaluate the segmentation and localization performance of LICO for a more comprehensive comparison with other explanation methods. (4) Some basic experiment settings, such as the datasets and base backbones, are expected to be explained clearly.

Questions

(1) The total loss function in algorithm 1 is inconsistent with that in Eq. (8). The Loss ‘LKL’ in algorithm 1 may need to be changed to ‘LMF‘. (2) Table 2 shows a decrease in classification accuracy for the base model ResNet when combined with GCC and CGC. The previous work cited suggests that good explanatory methods often sacrifice discriminative abilities. It would be helpful to explain how the proposed methods exhibit both good explanatory and discriminative abilities. (3) In Table 5, experimental comparisons based on Lce should also be carried out in order to better investigate the effectiveness of ‘LMF’ and ‘LOT’. (4) What is the effect of MLP used in the text encoder? Since the predicted probability only relies on the trained image encoder and classifier during inference, does there exist another more efficient way of processing the language features?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

N/A

Reviewer tr3q6/10 · confidence 3/52023-07-05

Summary

This paper introduces Language-Image-COnsistent (LICO) to get better interpretation for classification using the Vision-Language model. The proposed framework uses a frozen text encoder and a trainable image encoder to encode text and image information. The text is composed of several trainable prompt tokens and the text label for image classes. Then, the manifold matching (MF) loss is used to align the image feature latent space with the text feature latent space. In addition, another Optimal Transport (OT) loss is used to build a fine-grained correlation between prompt tokens and image features.

Strengths

1. The added text encoder during training time can introduce new information from the text encoder to the image encoder, but won't influence the inference procedure. Therefore, the inference procedure will keep the same as conventional classification models. 2. The insertion and deletion tests are used to validate the generated model interpretations from the proposed method, and LICO outperforms previous interpretation methods such as GradCAM and RISE in most of the cases. 3. The authors conduct experiments on several image classification benchmarks. For ImageNet, LICO obtains higher accuracy than the baselines. For other benchmarks such as CIFAR and SVHN, LICO achieves better performances than the baselines using limited amounts of labels. For fine-grained benchmarks such as Aircraft, LICO also shows better performances under full/few-shot settings in most of the cases.

Weaknesses

1. Did you compare the training time with the baselines? The inference time will be similar, but the training process will involve an extra text encoder. Therefore, it may take longer time to train the model and more space as well. 2. In table 6, do you have the result for 0 learnable context tokens?

Questions

see weakness

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

yes

Reviewer V7Pm2023-08-16

I thank the authors for providing this informative rebuttals. Some of them have addressed my concerns, while there are a few I still hold further questions for: - On the architecture. I agree with the statement of "LICO's design is model-agnostic" in the sense of computation, however when it comes to actual results ResNet and ViT would function differently and such difference would sometimes be reflected in very different attention maps. One example is the 2nd row, Figure 4 in [1]. - On pure text embedding model. Word2Vec is a relatively old method because 1) it was not trained on a large-scale dataset in terms of today's standard, and 2) the model may not have sufficient capacity. In some way, the table R1 in rebuttal can be explained as the deficiency of W2V instead of text-only v.s. image-text alignment. Would be more interesting to learn about the performance when the text embedding comes from a stronger model, i.e. comparable to CLIP in terms of model capacity and training data scale. A thorough discussion regarding this point would actually help to improve the quality of this work. [1] https://arxiv.org/pdf/2207.09684.pdf

Authorsrebuttal2023-08-18

Concerns about Different Architectures and Pure Text Embedding Models

We appreciate your further feedback. We’d like to address your further questions point-by-point. **On the architecture.** (i) We agree with you that CNN and Transformers are different in visualizing attention maps, but they are similar in incorporating with LICO due to it only depends on the latent representations, i.e., the representations before the final classification head. (ii) LICO does not affect the calculation of attention maps (the last self-attention) in ViTs. For image encoder of ViTs, LICO can also guide the representations of class tokens through proposed $L_{\text{OT}}$ and $L_{\text{MF}}$. (iii) In this paper, we follow the previous studies of interpretation, which focused on interpretation methods based on simple CNN backbones. **LICO can effectively overcome their common difficulty of improving interpretability and classification performance simultaneously**. Hence, we will treat incorporating LICO with ViTs as our future work in that there needs more effort to investigate how to obtain more explainable self-attention of ViTs, suitable quantitative metrics for interpreting ViTs, whether there exists trade-off between interpretability and classification performance, etc. (iv) Thanks for recommending the publication [1], which is a wonderful work that utilized partial distance correlation (DC) to measure similarity of different networks. The DC is helpful for generating improved attention maps via conditioning on another network, which benefits from its beautiful properties of end-to-end optimization and measuring feature spaces of different dimensions. This work inspires us to consider the relationships among features of different models and to further facilitate interpretation studies. - [1] On the Versatile Uses of Partial Distance Correlation in Deep Learning, ECCV 2022. **On pure text embedding model.** In addition to Word2Vec (W2V), we further applied pre-trianed BERT to testify the effectiveness of LICO: (i) The pure language BERT [1], (ii) The text encoder of vision-language BERT, i.e., BERT-ALIGN, ALIGN [2] is also a framework that aligns image and language features in latent space, which differs from CLIP in training with a noisy image-text dataset that is larger than that in CLIP. For both BERT [1] and BERT-ALIGN [2], we take learnable class-specific prompts as the inputs of text encoders. Based on the Table R1, we further provide the Table R3 as follows. We can see that BERT [1] surpassed W2V and W2V-P due to the generalizability of stronger pre-trained model. However, BERT still cannot achieve better performances than CLIP. Furthermore, BERT-ALIGN performs competitive and even better than CLIP, which attributes to its larger training set with more noisy image-text pairs. Consequently, from the results in Table R3 and this paper, we conclude that LICO works better with those vision-language pre-trained text encoders. The pure text encoders like W2V, even the pre-trained BERT with frozen parameters, are inferior in image-text alignment in LICO due the pre-trained parameters are not sensitive to visual features. This deficiency may be addressed by utilizing some transfer learning and domain adaptation tricks. **Table R3**: Comparison of different text encoders on CIFAR-10 | Encoder | CLIP | W2V | W2V-P | BERT[1] | BERT-ALIGN[2] | None | | :--- | :---: | :---: | :---: | :---: | :---: | :---: | | Full | $\textbf{95.8}$ | 95.6 | 94.9 | 95.7 | $\textbf{95.8}$ | 95.6 | | 4000 | 81.5 | 81.0 | 80.2 | 81.3 | $\textbf{81.7}$ | 80.9 | - [1] BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, ACL 2019. - [2] Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision, ICML 2021. Based on your valuable comments and suggestions, in our future work, we will comprehensively discuss different architectures and pre-trained models and try to unify the interpretation of CNNs and ViTs.

Authorsrebuttal2023-08-21

Experimental results with Transformer-based model

To address the major concern on the backbone, we further trained a ViT-Base-16 network on ImageNet-1k dataset, and provided accuracy, insertion, and deletion in **Table R4**. We calculated the $L_{\text{MF}}$ and $L_{\text{OT}}$ between language tokens and representations of patch tokens and class token. For attention maps, we applied Grad-CAM in LICO-trained ViT-Base-16 through calculating gradients from outputs to the last attention layer of class token. **Table R4** further confirms that the transformer model with LICO not only **performs better** but also **gains better interpretability** than the one without LICO, which is in line with the finding for CNN-based backbone. In our future work, we will further develop more explainable decision clues for ViT by incorporating knowledge of LLMs into self-attention. **Table** **R4** Evaluation on ImageNet-1k using ViT-Base-16 | ViT-Base-16 | Accuracy$\uparrow$ | Insertion$\uparrow$ | Deletion$\downarrow$ | | --- | --- | --- | --- | | w/o LICO | 77.9 | 55.2 | 14.4 | | w/ LICO | **78.2** | **56.0** | **13.8** |

Authorsrebuttal2023-08-21

We are looking forward to your feedback

Dear reviewer V7Pm, Thanks again for all of your constructive suggestions, which have helped us improve the quality and clarity of the paper! Since the author-reviewer discussion period will end soon in a few hours, we appreciate it if you take the time to read our further response and give us some feedback. Per your two major concerns, **we have demonstrated the effectiveness of ViT and BERT in LICO framework**, and most importantly, **ViT with LICO also obtains improved classification performance and interpretability**. Please don't hesitate to let us know if there are any additional clarifications or experiments that we can offer. If our response resolves your concerns, we kindly ask you to consider raising the rating of our work. Thanks for your time and efforts! Best, Authors of Paper 6636

Reviewer rFY82023-08-17

Thanks for the authors' effort in providing the rebuttal, which clarified several of my concerns. It can be claimed, to some extents, that this submission introduces a novel concept of semantic information by effectively aligning the prompt token with the feature map, . However, one aspect still requires further clarification. It remains somewhat unclear whether the proposed loss function genuinely contributes to the enhanced alignment. Despite that the authors added more ablation experiments on the loss function, the experimental outcomes, notably the accuracy results, indicate that the impact of LOT and LMF on performance improvements has not been consistently effective. Therefore, it may be more suitable to keep the current rating.

Authorsrebuttal2023-08-17

Effectiveness of proposed loss functions

We appreciate your further feedback. We would like to address your concern about the effectiveness of proposed losses. First, we'd like to clarify again that the presented LICO aims to improve interpretation ability of CNNs while maintaining or enhancing the classification performance. This is essentially challenging as discussed in previous studies such as Grad-CAM, Score-CAM, GCC, and CGC: although improving qualitative (attention maps) and quantitative (Insertion and Deletion) results, they sacrifice the decrease of classification accuracy. As shown in Tabs. 2, 3, and 4, LICO is able to **consistently** improve classification performance even under limited data settings. Most importantly, in Fig. 1 and attention maps provided in the paper and supplementary material, LICO exhibits superior interpretation ability against baseline methods. Second, in Tab. R2, LCE + LMF + LOT consistently improves the model performance. For the results in Tab. 1, LICO obtains better insertion and deletion values than baselines, and in Tab. 2, LICO outperforms CGC and GCC in classification accuracy. In the following Tab. R3, LCE + LOT and LCE + LMF also outperform accuracy of CGC and GCC. Hence, LOT and LMF are effective in improving qualitative and quantitative results compared with baseline methods. **Table R3**: Ablation on $L_{\text{MF}}$ and $L_{\text{OT}}$ | Loss | Top-1 | Top-5 | Insert. | Delet. | | :--- | :---: | :---: | :---: | :---: | | $L_{\text{CE}}$ | 76.13 | 92.91 | 53.5 | **13.3** | | CGC | 74.60 | 92.24 | 52.2 | - | | GCC | 74.40 | 92.12 | - | - | | $L_{\text{CE}}+L_{\text{OT}}$ | 75.98 | 92.92 | 56.6 | 16.0 | | $L_{\text{CE}}+L_{\text{MF}}$ | 76.18 | 92.90 | 56.9 | 15.5 | | $L_{\text{CE}}+L_{\text{OT}}+L_{\text{MF}}$ | **76.27** | **92.99** | **57.1** | 15.1 | Lastly, we highlight the significance of LICO in terms of **harmoniously bridging the gap between better interpretability and competitive classification performance**. Particularly in some real applications like medical imaging and autonomous driving, LICO pioneers an effective way of explainable AI by applying knowledge of LLMs.

Authorsrebuttal2023-08-21

We are looking forward to your feedback

Dear reviewer rFY8, Thanks again for all of your constructive suggestions. Hope our previous response can address your concerns. Since the author-reviewer discussion period will end soon in a few hours, we appreciate it if you take the time to read our further response and give us some feedback. If our response resolves your concerns, we are wondering if you would like to re-consider the rating. Thanks for your time and efforts! Best, Authors of Paper 6636

Reviewer tr3q2023-08-19

Response

Thank the authors for providing additional results. The results for 0 learnable context tokens makes sense based on the explanations. After reading all other reviewers' responses, I agree with Reviewer V7Pm that the backbone should be further discussed as ResNet50/18 are both CNN-based models. Grad-CAM can be easily adapted to ViT, therefore I also want to see if the proposed LICO can still be useful for transformer-based models. Missing discussions for ViT actually limits the scope of this approach, and the paper will be more interesting if the authors could validate the proposed method on SOTA image classification models.

Authorsrebuttal2023-08-21

Experimental results with Transformer-based model

We thank the reviewer for the further feedback. To address the major concern on the backbone, we trained a ViT-Base-16 network on ImageNet-1k dataset, and provided accuracy, insertion, and deletion in **Table R3**. We calculated the $L_{\text{MF}}$ and $L_{\text{OT}}$ between language tokens and representations of patch tokens and class token. For attention maps, we applied Grad-CAM in LICO-trained ViT-Base-16 through calculating gradients from outputs to the last attention layer of class token. **Table R3** further confirms that the transformer model with LICO not only **performs better** but also **gains better interpretability** than the one without LICO, which is in line with the finding for CNN-based backbone. In our future work, we will further develop more explainable decision clues for ViT by incorporating knowledge of LLMs into self-attention. **Table** **R3** Evaluation on ImageNet-1k using ViT-Base-16 | ViT-Base-16 | Accuracy$\uparrow$ | Insertion$\uparrow$ | Deletion$\downarrow$ | | --- | --- | --- | --- | | w/o LICO | 77.9 | 55.2 | 14.4 | | w/ LICO | **78.2** | **56.0** | **13.8** |

Authorsrebuttal2023-08-21

We are looking forward to your feedback

Dear reviewer tr3q, Thanks again for all of your constructive suggestions, which have helped us improve the quality and clarity of the paper! We’d like to note that **the concern of evaluating LICO on ViTs, initially raised by reviewer V7Pm, has been confirmed addressed**. Specifically, the additional Table R3 about ViTs also verifies the major merit of LICO that simultaneously improves interpretability and classification performance, which is not shared in previous popular and widely used interpretation methods. Since the author-reviewer discussion period will end soon in a few hours, we appreciate it if you take the time to read our further response and give us some feedback. Please don't hesitate to let us know if there are any additional clarifications or experiments that we can offer. If our response resolves your concerns, we kindly ask you to consider raising the rating of our work. Thanks for your time and efforts! Best, Authors of Paper 6636

Reviewer tr3q2023-08-21

Response

I appreciate the additional experimental results provided by the authors and would like to increase my score accordingly.

Authorsrebuttal2023-08-21

Summary of rebuttal and discussion

Dear Area Chair and all Reviewers, We sincerely thank you for devoting your time to evaluating our submission and engaging in a valuable discussion with us. Your insights have greatly enriched our work. 1. We clarify that the main strength of our LICO is simultaneously improving the interpretability and classification performance of the target backbone model. 2. A critical concern of reviewers V7Pm and tr3q is whether the LICO works well on ViTs backbone. Per this concern, we have provided an additional experiment using the ViT-Base-16 backbone, and the results show that our LICO is also helpful for improving the interpretability and classification accuracy of ViT. 3. The experimental settings in our paper are consistently based on previous widely-adopted settings on the interpretability of CNNs like Grad-CAM. Therefore, we politely argue that though evaluating transformer-based models can expand the scope of our work, it does not affect the integrity and innovation of the entire article. However, we also agree that it is worth studying a unified approach for interpreting CNNs and ViTs in our future works. Consequently, we believe that our responses have solved all concerns of reviewers. Finally, we will make the following changes in the revised submission according to your suggestions, including: (i) add discussion of implementation and evaluations using ViTs (by **tr3q** and **V7Pm**), (ii) localization evaluations (Table R1, by **rFY8**), (iii) pure text embeddings (Tables R1, R2, and R3, by **V7Pm**), (iv) evaluations on frozen parameters of prompts (Table R1 by **r2cD**). Thanks for your time and efforts! Best, Authors of Paper 6636

Authorsrebuttal2023-08-21

We appreciate your feedback and will definitely include them in the next version.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC