Aggregate-and-Adapt Natural Language Prompts for Downstream Generalization of CLIP

Large pretrained vision-language models like CLIP have shown promising generalization capability, but may struggle in specialized domains (e.g., satellite imagery) or fine-grained classification (e.g., car models) where the visual concepts are unseen or under-represented during pretraining. Prompt learning offers a parameter-efficient finetuning framework that can adapt CLIP to downstream tasks even when limited annotation data are available. In this paper, we improve prompt learning by distilling the textual knowledge from natural language prompts (either human- or LLM-generated) to provide rich priors for those under-represented concepts. We first obtain a prompt ``summary'' aligned to each input image via a learned prompt aggregator. Then we jointly train a prompt generator, optimized to produce a prompt embedding that stays close to the aggregated summary while minimizing task loss at the same time. We dub such prompt embedding as Aggregate-and-Adapted Prompt Embedding (AAPE). AAPE is shown to be able to generalize to different downstream data distributions and tasks, including vision-language understanding tasks (e.g., few-shot classification, VQA) and generation tasks (image captioning) where AAPE achieves competitive performance. We also show AAPE is particularly helpful to handle non-canonical and OOD examples. Furthermore, AAPE learning eliminates LLM-based inference cost as required by baselines, and scales better with data and LLM model size.

Paper

References (70)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer ZqyB5/10 · confidence 4/52024-07-06

Summary

This paper proposes a new method to enhance the downstream generalization of CLIP by distilling knowledge from LLM- or human-generated text prompts. The proposed method involves training a prompt generator to predicts prompt embeddings (AAPE) based on images.

Strengths

1. The method aggregates and distills task-related knowledge from LLM- or human-generated text prompts, leading to better downstream generalization of CLIP. 2. The approach demonstrates good performance on various downstream vision-language tasks.

Weaknesses

1. The analysis on the aggregated text prompt is missing. There is no evidence to prove that Input-Adapted Prompt Aggregator can alleviate the influence from noisy text. 2. The evaluation efficiency may be significantly impacted by the need to connect predicted prompts base text features. 3. A minor error is found in line 181, where “a photo of a {class} + category type” should be “a photo of a {class}”.

Questions

Can Input-Adapted Prompt Aggregator correctly distinguish between noisy and accurate text prompts? For example, in fig1, the last text prompt in the natural language prompts (“This image is of a red Jeep Compass SUV from 2012.”) inaccurately describes the training image. What’s the attention score of this prompt during aggregation? FLOPs of the proposed method. How long does it take to evaluate the proposed method and CLIP on ImageNet1k dataset?

Rating

5

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

The analysis on the aggregated text prompt is missing. The proposed method should be applied on more vision-language models, such as SigLIP[1]. This limitation has been mentioned in the paper. [1] Sigmoid Loss for Language Image Pre-Training.

Reviewer yCM35/10 · confidence 4/52024-07-08

Summary

The proposed method leverages language priors for better downstream adaptation and generalization of CLIP [37], which is similar to CuPL [36] that utilizes prompts generated by a LLM (e.g., GPT-3) for zero-shot image classification using CLIP. Unlike CuPL, the proposed method aggregates multiple LLM-generated or human-generated prompts to construct an aggregated text feature. Using the aggregated text feature as a guidance, the authors adaptively transform CLIP image features to tackle few-shot image classification, image-to-text retrieval, image captioning, or VQA. A notable difference from CuPL is that the proposed method does not run an inference of LLM at test time.

Strengths

This paper has the following strengths: (i) The proposed method is timely, and seems to be effective and efficient. By aggregating text features produced from LLM-generated or human-generated prompts and then using the aggregated feature as a guidance, the proposed method could effectively leverage language priors for better downstream adaptation and generalization of CLIP. After the distillation of rich language knowledge during the training time, the proposed method does not need to run an inference of LLM at test time. (ii) The authors validate the effectiveness of the proposed distillation method for downstream tasks (few-shot image classification, image-to-text retrieval, image captioning, VQA). It demonstrates that the learned adaptive transformation guided by language priors actually improves the adaptation and generalization of CLIP for downstream tasks.

Weaknesses

This paper has the following weaknesses: (i) This paper lacks the justification of how the adaptive transformation of CLIP image features, guided by an aggregated text feature, overcomes the image-text modality gap which exists in CLIP vision-language space [A]. Despite the modality gap, the proposed method simply encourages transformed CLIP image features to be located nearby an aggregated text feature (using L2 distillation loss). [A] Liang et al., “Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning”, NeurIPS, 2022. (ii) The proposed method might not be able to improve the model robustness to distribution shifts which are difficult to be explained by natural language. For example, the Terra Incognita dataset [B] contains distribution shifts caused by different camera locations in wild environments. The widely used benchmark of in-the-wild distribution shifts, WILDS [C], also contains such distribution shifts. Since the proposed method leverages only language priors, it might not be good at handling those distribution shifts. [B] Beery et al., “Recognition in Terra Incognita”, ECCV, 2018. [C] Koh et al., “WILDS: A Benchmark of in-the-Wild Distribution Shifts”, ICML, 2021. (iii) This paper lacks experimental results obtained with different distillation loss coefficient values, although the distillation loss is the main contribution of this paper. According to L191-196, it simply states that the proposed model is not sensitive to different coefficient values. Without relevant experimental results, it is difficult to evaluate this statement.

Questions

(i) Despite the image-text modality gap in CLIP vision-language space [A], the proposed method simply transforms CLIP image features using an aggregated text feature. Does “h(x)” overcome the image-text modality gap? How do the authors deal with the modality gap? If the proposed method fails to overcome the modality gap, the authors should justify why the proposed method leads to better downstream adaptation and generalization of CLIP. [A] Liang et al., “Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning”, NeurIPS, 2022. (ii) According to L176-180, the authors claim that learning h(x) is much more parameter-efficient than learning a sequence of token embeddings. Is it true? For example, suppose that we have 16 learnable word tokens, where the dimension size of each word embedding is 512. In this case, there are 16 * 512 learnable parameters. In contrast, as described in L169-170, “h($\cdot$)” is a network which consists of two fully connected layers. Given that the dimension size of CLIP features is 768 or 1024, the network seems to have 2 * 768 * 768 or 2 * 1024 * 1024 learnable parameters. What does it mean that learning h(x) is much more parameter efficient than learning a sequence of token embeddings? (iii) To validate the statement in L191-196 (“Performance is found to be not very sensitive to the $\lambda$ value in a wide range.”), the authors need to provide experimental results obtained with different $\lambda$ values.

Rating

5

Confidence

4

Soundness

2

Presentation

3

Contribution

3

Limitations

Yes, in page 9 and 16.

Reviewer khEq5/10 · confidence 4/52024-07-08

Summary

The paper proposes a new prompt embedding named Aggregate-and-Adapted Prompt Embedding (AAPE), which improves prompt learning by distilling knowledge about more detailed descriptions of classes into prompt embeddings. Concretely, the “summary” prompt is obtained by aggregating diverse reference prompts. Then, the prompt generator is trained to produce a prompt embedding that stays close to the aggregated summary while minimizing task loss at the same time. From the experiments, the proposed AAPE shows good performance on diverse tasks such as few-shot classification, VQA, and image-to-text retrieval.

Strengths

- How to well train prompt embeddings is one of the important topics in fine-tuning Vision-Language Models. - The paper is well written. - The proposed approach shows good performance in multiple tasks.

Weaknesses

- More details on using the CLIP reward are required. From my understanding, there is another loss to enhance the CLIP reward $\texttt{CLIP-s}\left(\boldsymbol{x}, \boldsymbol{p}^\alpha \right)$. - It would be better to include the performance of the model with only Aggregate-and-Adapted Prompt Embedding (AAPE) $h\left(\mathbf{x}\right)$ in Table 1 without using text embedding $\mathbf{w}_i$ and projection $g$. This result clearly shows the performance gain induced by the Aggregate-and-Adapted Prompt Embedding. - It would be better if the efficiency comparison was included in the paper compared to other prompt learning methods such as CoOp, MaPLE, and PromptSRC, etc.

Questions

Please refer to the weaknesses.

Rating

5

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

The authors have adequately addressed the limitations and potential negative societal impact.

Reviewer khEq2024-08-12

Thank you for your response. Most of my concerns have been addressed. I have reviewed all the reviews and rebuttals. However, I still have a few questions: - What is the computational cost during the training phase? - I'm still curious about the performance of the proposed method with no $w_i$ and $g$ on the classification task. I think that the proposed method can perform well on at least base (seen) classes without using $w_i$ and $g$. Thanks.

Authorsrebuttal2024-08-13

Re: Official Comment by Reviewer khEq

**Computational cost for training** Thanks for the reminder of adding such info. Here we show the training cost in terms of GFLOP/time (min). We mainly compare AAPE (162.6/41.92) with CoCoOp (162.5/39.53) since both methods learn input-adaptive text prompts, only that AAPE incurs a small overhead for learning an additional prompt aggregator. When compared to CoOp (162.5/10.08) and PromptSRC (179.6/13.13), AAPE is less time-efficient but has comparable or better GFLOP. **Classification performance of AAPE $h(x)$ without $w_i$ and $g$** Thanks for providing the insights. We have finished testing an $h(x)$-only baseline for classification, i.e., using $h(x)$ in place of $x$ to act as a proxy image query, while the text classifier is the basic template $w_i$ that allows us to encode different class names to perform classification. This simple setting is similar to that of the image-to-text retrieval task, both of which can be viewed as the testbeds for the text knowledge captured in $h(x)$. | Method | Base | New | H | | -------- | ------- | ------- | ------- | | AAPE (default) | 84.72 | 77.54 | 80.97 | | $h(x)$ only | 84.01 | 75.93 | 79.77 | | CuPL | 74.31 | 75.25 | 74.78 | | AAPE w/o $L_{distill}$ | 79.47 | 73.25 | 76.23 | | CoCoOp | 80.47 | 71.69 | 75.83 | We see from the table that our $h(x)$-only baseline actually performs well, on both base and new classes. This indicates the good generalization using an input-adapted "captioning latent" $h(x)$ to distinguish different classes. Our default approach combines $h(x)$ and the template $w_i$ that contains explicit class name information (using projection $g$). By doing this, we achieve better performance than $h(x)$-only, and in the meantime, enable easy interpretation of the roles of $h(x)$ and $w_i$ during classification. When compared to CuPL that simply ensembles LLM-generated text prompts for classification, our $h(x)$-only baseline outperforms significantly by learning input-adapted prompts. As a reference, we also compare with the methods of AAPE w/o $L_{distill}$ and CoCoOp, which both predict input-adapted text prompts but without language supervision. The benefits of our $h(x)$-only baseline are evident, thanks to the help of the learned language priors. Will add the results in final paper.

Reviewer 7B5Q5/10 · confidence 3/52024-07-11

Summary

This framework first aggregates textual knowledge from human or large language model (LLM) generated prompts into a summary aligned with each input image. This is achieved using a prompt aggregator. A prompt generator is then jointly trained to create prompt embeddings that are close to this aggregated summary while also minimizing task-specific loss. The method demonstrates improvements in performance on various downstream vision-language tasks, including few-shot classification, visual question answering (VQA), and image captioning without incurring LLM inference costs during testing.

Strengths

The motivation is clear and the method is reasonable to me. The proposed method extends the application of downstream tasks from the common image classification to image-text retrieval, visual question answering (VQA), and image captioning. The performance on Flickr30k dataset is remarkable.

Weaknesses

This paper misses one relevant research in prompt learning “ArGue: Attribute-Guided Prompt Tuning for Vision-Language Models.” ArGue introduces an attribute-guided prompt tuning approach that outperforms traditional prompt learning methods in specific tasks and datasets. The failure to reference and compare these latest methods may limit the comprehensiveness and advancement of the proposed method. The comparison methods used for the image-to-text retrieval task in this paper are somewhat outdated and do not incorporate the latest research advancements. For example, BLIP-2, as a new image-to-text retrieval method, has demonstrated superior performance across various tasks and datasets. To more accurately assess the effectiveness of the proposed method, the latest comparison methods such as BLIP-2 should be included in the experiments. The method in this paper shows overfitting on the base classes, with significantly lower performance on the novel classes compared to the base classes. This indicates a lack of generalization ability when dealing with new categories, potentially leading to poor performance on unseen data in real-world applications.

Questions

In Figure 1, what input is provided to GPT-3? Is it only text? Why does the last sentence of the description contain "the image is..."? Where are human-generated image captions obtained from?

Rating

5

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

yes

Reviewer UfdC6/10 · confidence 4/52024-07-13

Summary

The paper proposes a prompt learning method that distills the knowledge from pre-trained LLM while conditioning them on the image embedding. A Prompt Aggregator module combines the LLM generated prompts per image and the Prompt Generator module, generates a prompt from the image embedding. The modules are trained using a regularization loss between the aggregated and generated prompts, and the downstream task loss. Experimental results on 11 image classification datasets, four imagenet domain variants, two image captioning datasets and VQA dataset show the effectiveness of the proposed prompt learning method.

Strengths

- The experimental results cover a broader spectrum that includes image classification, vision-language understanding, such as captioning and VQA. The results show the proposed method is effective across the range of tasks.

Weaknesses

- The method is shares a lot if similarity to [1] which distills the LLM knowledge through prompts to the CLIP text encoder. It is important to compare and distinguish how this work differs from [1]. - See questions [1] Khattak, Muhammad Uzair, et al. "Learning to Prompt with Text Only Supervision for Vision-Language Models." arXiv preprint arXiv:2401.02418 (2024).

Questions

- What are the number of parameters learnt in comparison to other prompt learning methods? Is the final projection layer necessary? - The zero shot performance on the captioning task seems promising. Is there a projection layer $g$ for the VQA and captioning tasks as well? Perhaps the projection layer being trained can be giving a significant boost in the classification task. What are the authors thoughts on this?

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

none

Reviewer yCM32024-08-10

Dear Authors, Thanks for the response! Most concerns were successfully resolved, **except whether AAPE $h(x)$ overcomes the image-text modality gap**. Do the authors think **AAPE $h(x)$ would be located nearby text features** rather than image features in Figure 1 of the paper [A]? (not between text and image features) - [A] Liang et al., “Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning”, NeurIPS, 2022. I'm still not convinced whether $h(\cdot)$, which consists of two fully connected layers, could fully overcome the image-text modality gap. If the authors claim that the **lightweight module (i.e., two fully connected layers) is sufficient for overcoming the image-text gap by simply using L2 distillation loss**, then they need to provide validation results such as Figure 1, 4, or 8 in the paper [A]. If it is true, then the discovery would be helpful for understanding why the proposed method works well in general. Best regards, Reviewer yCM3

Authorsrebuttal2024-08-13

Re: Official Comment by Reviewer yCM3

Thanks for sharing this paper [A]. After a careful read of [A], we realize that we mistook the concept of "modality gap" for "image-text alignment" in terms of the (cosine) feature similarity. Based on the cosine similarity, according to our previous response, AAPE does improve this metric (e.g., 0.91 vs. 0.89 of CLIP on ImageNet) mainly because $h(x)$ is an explicit image-to-text mapping from image feature $x$. On the other hand, [A] defines the modality gap as the Euclidean distance between **the centers** of image and text features, and shows that contrastive learning preserves modality gap. Here we provide some new analysis and empirical results for AAPE based on such defined modality gap. **AAPE does not overcome the modality gap.** More accurately speaking, AAPE may sometimes reduce the gap distance after prompt tuning, but only by a moderate amount (e.g. from 0.82 to 0.78 on ImageNet). This is because we do not train to reduce the gap. Instead, AAPE is optimized by two loss terms: 1) L2 distillation loss that makes $h(x)$ located near the aggregated text embedding $p^a$, 2) contrastive task loss for classification which maximizes the image-text feature similarity again. This is definitely different from those experiments in [A] that either manually shift the image and text features to explicitly close the gap (Section 4.2), or directly transform text features to be close to image feature (Section 4.4). **Then why does AAPE generalize without closing the modality gap?** One key argument in [A] is that it’s not clear that it’s desirable to reduce modality gap for improving downstream performance. Interestingly, the authors found a larger gap can help some zero-shot learning and fairness tasks, while other tasks benefit from a smaller gap. In our case, we similarly found good generalization does not necessarily need a reduced modality gap by e.g., pulling AAPE back to the neighborhood of image features. To explain AAPE's good generalization, we previously rely on intuitive reasoning that AAPE introduces rich language priors, and that AAPE promotes image-text alignment. Inspired by [A], here we attempt to go from the optimization perspective. Our hypothesis is that we have a reshaped loss & accuracy landscape when optimizing AAPE with a multi-task loss (L2 distillation + contrastive) rather than contrastive loss alone. We will add visualizations of our reshaped loss/accuracy landscape w.r.t modality gap (similar to Fig.3 and 10 in [A]), where the optimal gap is different from the gap of pretrained CLIP. Generally speaking, the optimal modality gap minimizes 2 different objectives in our multi-task loss which could move in opposite directions: the contrastive loss learns AAPE to (over)fit the seen classes (a common belief in the literature), while the L2 distillation loss moves AAPE closer to $p^a$ with external text knowledge (thus modifying the image-text modality gap) which often benefits generalization to unseen classes as evidenced by our results. It would be an interesting direction for future research to study both 1) how our multi-task optimization dynamics could affect the modality gap and 2) the relation between the gap distance and downstream generalization. We plan to include the above analysis in Appendix. We also want to add that AAPE is designed to be a general image captioning latent. This makes AAPE easily go beyond few-shot classification and generalize to more vision-language tasks. For better generalization in large-scale tasks, on-going investigations include scaling up the training data and model size of $h(\cdot)$ (beyond two fully connected layers).

Reviewer yCM32024-08-13

Thanks for the detailed response! **As acknowledged by the authors, it seems that $h(\cdot)$ does not overcome the image-text modality gap.** From this observation, what I wonder is why AAPE $h(x)$ works well in general. If my understanding of the proposed method is clear, AAPE $h(x)$ should be treated as a kind of text features (rather than image features, although they are obtained via a transformation from image features). **If not, L2 distillation loss does not make sense due to the modality gap.** Could the authors elaborate them in this perspective?

Authorsrebuttal2024-08-13

Re: Official Comment by Reviewer yCM3

Thanks for your fast feedback, and yes, AAPE is a text feature vector. Since AAPE is generated from input image features and it's regularized by LLM-generated text prompts via a distillation loss, we actually view AAPE as a latent vector for image captioning. From our previous response, we have detailed the reason why AAPE works. As a quick recap, AAPE learning is governed by two signals: distillation loss that equips AAPE with the captioning capability which could generalize to describe both seen and unseen image classes; and contrastive loss that allows AAPE to (over)fit the seen classes. Optimizing such multi-task loss may lead to an increased or decreased image-text modality gap (depending on the image & prompt distributions for tuning), whereas AAPE is shown to consistently achieve strong generalization performance. This is yet another proof of the main argument in [A]: good generalization does not necessarily need a reduced modality gap.

Reviewer yCM32024-08-13

Thanks for the response! Although I acknowledge the claim in the paper [A] (good generalization does not necessarily need a reduced modality gap), the claim is still **limited to classification**. The modality gap is essentially caused by the temperature scaling used in the contrastive loss for classification, and I agree that such modality gap sometimes could be useful and even ignored in classification as demonstrated in the paper [B]. - [B] Zhang et al., "Diagnosing and Rectifying Vision Models using Language", ICLR, 2023. However, I'm **not convinced that such modality gap is useful in general (not only for classification)**. Unlike classification where the modality gap still preserves cosine similarity ranks, AAPE $h(x)$ should be treated as a kind of text features in this paper. **It seems that this paper lacks justifications and the proposed method just surprisingly works well in general.** Could the author provide more justifications from this perspective?

Authorsrebuttal2024-08-13

Re: Official Comment by Reviewer yCM3

Some clarifications: - From our new empirical studies, we **do not claim that Modality gap is useful**. Instead, our main observation is that the gap is not highly correlated with generalization performance. We observed that after prompt tuning, the modality gap is reduced on 7 out of 11 datasets and moderately increased on the remaining 4, while the performance is consistently improved. This gives rise to our argument that "good generalization does not necessarily need a reduced modality gap" which is in line with [A]. To be fair, we **also don't claim we fully overcome** the modality gap with AAPE learning. - One key reason for AAPE's good generalization is the strong image-text alignment as measured by cosine feature similarity. This is achieved by our image-to-text mapping function $h(\cdot)$, which encourages feature alignment with increased similarity score as given before. Hence, AAPE $h(x)$ is actually a text embedding that **preserves the cosine similarity notion, which could explain the reduced modality gap on 7 classification datasets**. - From the optimization perspective, our good generalization is also attributed to the multi-task learning objective: the distillation loss introduces language priors into AAPE, which promotes generalization and avoids overfitting from contrastive loss. Note such multi-task learning shifts the modality gap differently for each dataset, but all shifting in the directions (on loss landscape) that favor loss decrease and thereby performance gains. This observation under the multi-task learning framework **can generalize beyond the classification task**, since it **does not require varying the contrastive loss's temperature** which is limited to classification. - **For tasks of image captioning and VQA**, we similarly optimize a multi-task objective on COCO dataset to learn AAPE: distillation loss + contrastive loss for image-text retrieval. Our latest evaluation on the image-text modality gap shows the gap is actually reduced from 0.79 to 0.72 after AAPE learning. This may be a signal that **our multi-task loss can both mitigate modality gap to some extent and achieve competitive performance**. We just need to verify this hypothesis on more tasks/data distributions, which may provide hints on the relation between modality gap and downstream generalization in complex vision-language tasks. We will provide justifications in Appendix about why AAPE generalizes, as briefly summarized below: - Generalization is not strongly correlated to the modality gap reduction for classification, with empirical results. - Reasoning of image-text alignment and multi-task learning that promotes generalization. To offer empirical support by showing the feature similarity score and multi-task loss landscape vs. modality gap, respectively. - Evaluating how the hypothesis of "multi-task loss may mitigate modality gap" extends to other vision-language tasks, with preliminary results.

Reviewer UfdC2024-08-12

Thank you for the response to the comments. I have read all the reviews and the rebuttals. My concerns have been addressed sufficiently and I will raise my rating.

Authorsrebuttal2024-08-13

Re: Official Comment by Reviewer UfdC

Thanks for raising the score! We promise we will integrate all the new discussions and results in the final paper.

Reviewer ZqyB2024-08-13

I have read the author's response and maintain my rating.

Authorsrebuttal2024-08-13

Re: Official Comment by Reviewer ZqyB

Thanks for the feedback. We will continue to improve the paper and integrate all the insights from discussions with different reviewers.

Reviewer 7B5Q2024-08-13

Reply to rebuttal

I read all the reviews and rebuttals, and decide to maintain the original rating.

Authorsrebuttal2024-08-13

Thanks for the feedback. We will continue to improve the paper and integrate all the insights from discussions with different reviewers.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC