Bootstrapping Vision-Language Learning with Decoupled Language Pre-training

We present a novel methodology aimed at optimizing the application of frozen large language models (LLMs) for resource-intensive vision-language (VL) pre-training. The current paradigm uses visual features as prompts to guide language models, with a focus on determining the most relevant visual features for corresponding text. Our approach diverges by concentrating on the language component, specifically identifying the optimal prompts to align with visual features. We introduce the Prompt-Transformer (P-Former), a model that predicts these ideal prompts, which is trained exclusively on linguistic data, bypassing the need for image-text pairings. This strategy subtly bifurcates the end-to-end VL training process into an additional, separate stage. Our experiments reveal that our framework significantly enhances the performance of a robust image-to-text baseline (BLIP-2), and effectively narrows the performance gap between models trained with either 4M or 129M image-text pairs. Importantly, our framework is modality-agnostic and flexible in terms of architectural design, as validated by its successful application in a video learning task using varied base modules. The code will be made available at https://github.com/yiren-jian/BLIText.

Paper

References (75)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer FD1R6/10 · confidence 5/52023-06-14

Summary

This paper propose to add a new component, a P-Former, to the BLIP-2 vision-language pre-training framework. The P-Former is a sentence encoder that learns to project texts into the input space of a LLM. During vision-language pre-training, an additional alignment loss is applied between BLIP-2 Q-Former's visual feature and the P-Former's text feature, which improves the alignment between the Q-Former and the frozen LLM, leading to improved image-to-text generation performance.

Strengths

- This paper explores an interesting direction to improve BLIP-2 pre-training with an additional text encoder. The text encoder serves as a surrogate of the LLM to help better aligning the Q-Former with a frozen LLM. This could open up future research opportunities. - The experiments show some good improvements w.r.t. the original BLIP-2 on image-to-text generation tasks, which verifies the improved alignment with the LLM. It is also good to see improvements with both OPT and FlanT5.

Weaknesses

I enjoy the core idea of the paper. However, the paper still needs much improvement in multiple aspects as detailed below. 1. The motivation and problem formulation is a slight misrepresentation of the actual method. The actual role of the P-Former is to serve as a surrogate of the LLM to help the Q-Former better align with the frozen LLM in both stage-1 and stage-2 pre-training. 2. The proposed method has a significant conceptual difference from BLIP-2: the stage-1 pre-training is *not* LLM-agnostic. This means that the more expensive stage-1 pre-training needs to be performed for every LLM. The proposed method thus tradeoff the Q-Former's flexibility for better alignment with the LLM. This point needs to be highlighted. 3. The pre-training of the P-Former is rather complex, where the intuition of each training loss is not clear. There needs to be more discussion and ablation study on the multiple losses. 4. The effect of the P-Former alignment may or may not maintain given more pre-training data and longer pre-training epochs. This is hard to verify given limited compute, so I'm not expecting the authors to provide answers. 5. The writing could be improved. Personally, I feel many equations to be unnecessary because they add additional burdens to understand the method. Simple plain text would be good enough to describe the method clearly. The "forward-decoupled" and "backward-decoupled" concepts are also not intuitive to understand.

Questions

- Is the P-Former finetuned during stage-1 and stage-2 pre-training?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

The authors have addressed the limitations.

Reviewer T7A17/10 · confidence 4/52023-07-05

Summary

This paper introduces a vision-language pre-training method with the help of the proposed Prompt-Transformer. As the first step, the P-Former is optimized to learn the "optimal" soft prompt that can guide the LLM to generate the target texts. After that, the trained P-Former is frozen and used to train the visual adaptor of VLM.

Strengths

1. The method is novel in that it utilizes an intermediate P-Former to effectively optimize the VLM. 2. The method achieves impressive results in terms of the performance of training data efficiency.

Weaknesses

1. In Table 1, it is not fair to only list the numbers of image-text pairs considering that P-Former also consumes language data for training. Similarly, in the paragraph of Line 216, the overhead of training P-Former should also be taken into consideration. 2. After reading the paper, it is still a bit unclear why, in essence, P-Former is effective. Does it function as some type of knowledge distillation? Please provide more discussion about this.

Questions

None

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

4 excellent

Limitations

The authors discussed three limitations in the paper. I am mainly concerned about the last one that the method cannot handle cross-attention models. With the development of VLM, cross-attention or joint modeling could be a very important branch of methods.

Reviewer 8epX5/10 · confidence 5/52023-07-07

Summary

This paper introduces a prompt-transformer (P-Former), which is pretrained on text corpus, that can trigger the LLM to generate better text prompts to align with the vision-and-language models with better visual features. Empirical experiments on the top of BLIP-2 shows promising results on several tasks.

Strengths

1. This paper introduces an separate stage for P-Former, to trigger LLM to generate text prompts to align with visual features, verifies the ideas on the BLIP-2, and shows promising results on several tasks.

Weaknesses

1. May need more comprehensive experiments to show where is the gain from.

Questions

1. Where is the gain from? I assume the stage 1 in the Figure 3 and Line 211, is just taken from the pretrained BLIP model, right? If not, do you take pretrained BLIP checkpoint for continuing pretrain 10 epochs? And do you have a comparison for the stage 1 pretraining, compared to the BLIP-2? The main issue here is, to try to make an alignment with BLIP-2 to see where the difference/improvement comes from? stage 1 or stage 2. 2. Is the P-Former is agnostic to the different Vision-encoders? if changing a vision encoder, does the framework still work?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

2 fair

Presentation

2 fair

Contribution

2 fair

Limitations

See the questions part.

Reviewer jFy56/10 · confidence 3/52023-07-09

Summary

To improve image-to-text generation, this paper proposes a proxy model P-Former to predict LLM prompts and uses it as an auxiliary loss in BLIP-2 to align selected features with LLM prompts. Results show promising results, especially in 0-shot VQA tasks.

Strengths

Nice novelty by introducing a proxy model for LLM prompts prediction to enhance image-to-text generation. The experiments, ablations and analysis are comprehensive, showing improvements in both VQA (more significant) and captioning (less significant) tasks, and no improvement in image-text retrieval. Well written and easy to read.

Weaknesses

This paper only shows the effect of the LLM prompt prediction in an incremental way, i.e., as an auxiliary loss in the existing BLIP-2. It would be more interesting if we could show the effectiveness of P-Former in a cleaner (simpler) setup. minor: The paper title says “Bootstrapping Vision-Language Learning”, which might be generic according to the results of this paper, such as no improvements on image-text retrieval tasks.

Questions

As mentioned in “Weaknesses”, instead plugging P-Former in BLIP-2 as an auxiliary loss (IMO more like an incremental work), it might be more interesting to verify P-Former’s effect by directly predicting the LLM prompts in a captioning model? In line 216, also mention the cost to train P-Former? In line 196, elaborate the size of each dataset? minor comment: in equation 7, the parentheses seem incorrect?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

No specific concern.

Reviewer 3ni96/10 · confidence 4/52023-07-27

Summary

The paper introduces a novel approach for optimizing the application of large language models in resource-intensive vision-language pre-training. Unlike the traditional approaches of using visual features as prompts to guide language models, the paper focuses on identifying optimal prompts to align with visual features. This is achieved through the introduction of the P-Former, a module trained only on linguistic data, eliminating the need for image-text pairings.

Strengths

1. Originality: The paper demonstrates a high level of originality in its approach to vision-language learning. By introducing the P-Former, the paper presents a unique perspective in contrast to the standard approach of using visual features as prompts. Also, the focus on a modality-agnostic framework expands the applicability to various VL tasks, further enhancing the originality of the paper. 2. Quality: The paper exhibits a good level of quality in various aspects. The methodology is well-formulated, and the introduction of P-Former is well-justified. Also, it provides sufficient details about the experiments and demonstrates the robustness and flexibility of the framework across different VL tasks. 3. Clarity: The paper is written with clarity and precision. The abstract, introduction, and related work sections provide a clear context and properly position the paper in the literature. The technical concepts are well-explained, including the P-Former and the decoupled language pre-training approach. The use of figures and tables further help in understanding the experiments and results. 4. Significance: The significance of the paper is in the decoupled language pre-training approach, with the P-Former's ability to predict ideal prompts, addressing resource-intensive challenges in this field. Also, it offers a fresh perspective on how to leverage large LMs in VL learning scenarios.

Weaknesses

- The idea of the P-Former is interesting, however, it seems to lack an intuitive explanation and motivation. Why learning an ideal language prompt helps more, compared to using visual ones as in the counterpart models? - There seems to be a lack of some ablations, which naturally arise as questions, for example, experimenting with and without the P-Former module, maybe just by using randomly initialized prompt p, using different sizes/types of language backbones, different ways of initializing the prompt p at the beginning, etc. - Also, there is a lack of qualitative analysis of the experiments. I would recommend including and analyzing qualitative results in comparison to existing approaches. Presenting visual examples of the model's performance in both successful and failure cases can make the paper stronger. - Some experiments lack stronger interpretation. I would also encourage the authors to provide more interpretation of some results (e.g. tables 1, 2, 3), rather than describing the tables, to enable the readers to gain better insights into the behavior of the framework.

Questions

- Could the authors provide more details and clearer insights into the P-Former part of the framework and size? - Across all experiments, it can be seen that more data helps BLIP-2 to outperform the P-Former. I wonder if the P-Former will improve the best performance of BLIP-2 if the pre-training image-text data is larger (129M)? - In Table 3, it can be observed that the performance of the proposed model is lower when it comes to retrieval. Could you interpret these results and elaborate more on this limited performance? - What is the reasoning behind the need for a separate pre-training of the P-Former on language data? Why do additional unimodal sentences contribute to the performance in Table 6? - Since the framework is presented as modality agnostic, do you think it can handle language-only tasks, such as machine translation? If so, which parts of the framework would need to be adapted to handle this?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The authors have provided a sufficient discussion of the limitations of their work in section 5. Due to the usage of pre-trained backbones, I would also encourage the authors to discuss any potential negative societal impact of their work.

Reviewer FD1R2023-08-13

I appreciate the authors' response. I confirm my original score and recommend acceptance.

Authorsrebuttal2023-08-14

Thank you!

Thank you! We appreciate the time and effort.

Reviewer 8epX2023-08-17

Thanks for the authors' response. May lend to recommend a weak acceptance, either 5 or 6.

Reviewer jFy52023-08-17

> Given the end goal is caption/text generation, the LM loss remains a pivotal factor for ensuring the proficiency of such a model. IIUC, this is done by the frozen LLM instead of Q-Former? Given the input of the LLM are tokens, can we train a clean generative model to predict these tokens (generated by the pre-trained P-Former). It's fine to reuse Q-Former's architecture and training recipe, but my concern is that why we couldn't use predicting P-Former tokens as the only objective (I assume such a generative model is not that large, unlike the frozen LLM, so maybe this is still doable)?

Authorsrebuttal2023-08-18

Theoretically, it is feasible to train our generative model (ViT+Qformer), denoted as `Gen()`, with the singular objective of predicting the prompt `prompt_gt` generated by the P-Former: ``` prompt_gen = Gen(Image) L = (prompt_gen - prompt_gt) ** 2.0 ``` Nevertheless, driving this loss to an exact zero is an unattainable goal. Significantly, **any deviation in learning the prompt is amplified when the prompt is subsequently passed into the LLM** given that generation is our end objective: ``` caption_gen = LLM(prompt_gen) ``` Even a minor discrepancy in `prompt_gen` can disproportionately affect the accuracy of `caption_gen`, especially when processed through an intricate LLM. The suggestion by the reviewer to rely solely on the objective of prompt alignment loss `L = (prompt_gen - prompt_gt) ** 2.0` is theoretically valid. However, our empirical studies indicate that it fails to deliver the most desirable outcomes: Specifically, when testing the alignment loss in isolation (without the LM loss) on the VATEX dataset for video captioning, we observed scores of CIDEr: 35.1, BLEU-4 19.8, and ROUGE: 42.1. These results accentuate that our P-Former can adeptly guide vision encoders along favorable trajectories even in the absence of the LM loss. Nevertheless, these scores are inferior to those reported in Table 7. One primary reason is that relying only on the prompt alignment loss excludes both the forward and backward passes in the computationally intensive LLM during training. It's worth noting that the LLM represents over 70% of the parameters in the entire model of BLIP-2 ViT-g OPT2.7B (and surpasses 90% in BLIP-2 ViT-g FlanT5XXL). This not only pertains to a performance-tradeoff but also to the cost of training. While our model demonstrates promising results, it cannot replicate these results without the incorporation of the LM loss. Herein, our prompt alignment loss serves as an intermediary, aligning the Q-Former more cohesively with the frozen LLM. We concur with the sentiment that a learning framework that circumvents the LLM, yet maintains robust performance, would indeed be groundbreaking due to the substantial reductions in training duration and computational demands.

Reviewer jFy52023-08-18

> L = (prompt_gen - prompt_gt) ** 2.0 I thought the input for the frozen LLM is **tokens**, but from the formula above, we are actually using **embeddings** as the input -- yeah, this would be hard for the Q-Former to learn indeed; also, it seems too difficult or impossible to learn prompt tokens instead of prompt embeddings for P-Former?

Authorsrebuttal2023-08-19

You are right in observing that the input to the LLM is based on soft prompts, which are, in essence, embeddings. To align with terminologies often used in literature, the terms "soft-prompts" and "visual-tokens" sometimes both refer to embeddings, especially in the context of VLM, distinguishing them from traditional text-tokens. Our decision to use soft-prompts in our framework is rooted in two core considerations: - **Compatibility with Established Approaches**: Several significant efforts in the domain, such as BLIP2, Frozen, ClipCap, X-VLLM, and even more recent endeavors utilizing LLaMA, have all adopted **soft-prompts** (embeddings) as the primary input for the LLM. In choosing this path, our objective was to ensure that our methodology was in harmony with current best practices in the field. - **Challenges with Text Tokens**: To the best of our knowledge, there haven't been prominent studies that use text tokens (i.e., discrete integer indices) as prompts for LLM in end-to-end VLM systems. One primary reason for this might be the challenge of differentiability. When utilizing text tokens, one would need to apply an 'argmax' operation on the output from the Q-former (or similar VL connectors) before introducing it to the LLM. Although there are methods like gumbel-softmax, vector quantization, or stop gradients that potentially bypass this obstacle, the intrinsic optimization challenges these present might be why contemporary research consistently veers towards the use of soft-prompts (or embeddings) as the input for LLM. Basically, while the idea of using token indices might seem more straightforward, the complexities it introduces and the desire to maintain consistency with existing works led us to opt for embeddings as the more pragmatic choice for our model.

Reviewer jFy52023-08-19

Thanks for the clarification, which has addressed all my concerns!

Reviewer T7A12023-08-18

Thanks for the reply. My concerns are basically addressed.

Reviewer 3ni92023-08-21

Thanks for the response, it clarified all my concerns. I would recommend acceptance.

Authorsrebuttal2023-08-21

Thank you!

We greatly appreciate the reviewer's confidence in our work.

Program Chairsdecision2023-09-21

Decision

Accept (spotlight)

© 2026 NYSGPT2525 LLC