Implicit Multimodal Alignment: On the Generalization of Frozen LLMs to Multimodal Inputs

Summary

The authors propose an exploration of the Implicit Multimodal Alignment (IMA) phenomenon in frozen Large Language Models (LLMs): when exposed to perceptual tokens (e.g. from image or audio features), authors show that those tokens are implicitly aligned on text tokens, even so the LLM has only been trained on text previously. The study argues this phenomenon is a fundamental mechanism into LLMs, and that it is relevant to (i) better understand task performance (e.g using that implicit alignment as proxy) and (ii) to better understand model visual faithfulness (i.e absence or presence of hallucinations). Based on their findings, the authors propose as well architecture changes such as skipping specific operations on perceptual tokens in order to decrease inference cost.

Strengths

- This is a very original and in-depth work to understand the inside mechanisms appearing in MLLMs. Many/Most of the current published MLLMs are based on variations a pre-trained LLM + a pre-trained vision encoder with smaller VL connector in between. The insights from this study could be applied on many of those works, especially to better improve the visual faithfulness of those models.

Weaknesses

- This is not commonly an important weakness but for this work, it might be: the presentation of the results and the overall quality of the layout organization is hurting the understanding. All the figures are pixelated bitmap/jpeg screenshots and are barely readable. This doesn't change the pertinence of the work, but hurt importantly how well its content is conveyed. I urge the authors to revisit this. Many solutions exist today, such as TKIZ or exporting figures in vectorized PDF to better include them in the final document. Lacking of space in a limited-page paper is understandable, but not using TKIZ or vectorized figures is not. Figure 5 for instance shouldn't be as is in this work. I challenge other readers to read/decipher the labels of the x and y axes on the heatmaps. - In many details, the work could have be improved. See the "Typos" list below. As well, some claims are not backed anywhere. For instance, Figure 1 and Figure 12 both claim a 70% sparsity. Where is that introduced in the main text? Consistency is not respected as well: in some contexts, P means Prompt while in other context means Perceptual. Is Prompt == Perceptual? This is not stated in Section 2. Prompt usually refers to the text and image prompts, before preprocessing and tokenization. The formatting is off as well. Sometimes, $P$ is used, sometimes plain-text P. Same for $T$ vs. T. - Evaluation could be more complete for the Multi Task setup with the use of common benchmarks such as LLaVABench-in-the-Wild, MMVet, etc. As well, Hallucinations benchmarks such as MMHALBench and HallusionBench should be used. Authors quote [46], which is the very work that introduces MMHALBench. Typos: - Line 90: Define k? - Line 94: FC1 and FC2 should be $FC1$ and $FC2$. Same for g, LN1, LN2, SA. - Line 92: What is (B)? It is not defined in eq (2). - Line 101: s/We/we/ - L105: Reformulate "In the paper, we focus no LLaVA-1.5-4 as it is most similar to the ST setup, and analyse other variants in App. E."? s/ no / on / ? - L106: (e.g. Sec. 3) ? Do you mean (see Sec. 3)? Why e.g.? - Section/Equation/Appendix references are not following the same format. Sec. is used in place of Sec., same for App. and Appendix, but Equation is fully written. What about adopting a consistent terminology? - Figure 2 is barely readable, same for Figure 3, Figure 4. Figure 5, as well it is not vectorized. Consider using TKIZ or export your figures in PDF and include those cropped PDF in your tex document. Bitmap/JPEG/PNG used here are leading to poor and pixelated results, hurting the readability. - Figure 1: s/Analysing/Analyzing/ - Figure 1: s/LMM/LLM/ (twice) - Figure 1: s/perceputa/perceptual/ - Figure 1: "Computation efficiency", do you mean "Computational efficiency" or "Computation and efficiency"? - Figure 1: What is the point of the right block named "Framework"? It doesn't seem related to the work or the caption of Figure 1. - L136: The claim "Fig. 2 shows a clear narrow cone effect for textual and perceptual tokens." is not verifiable when looking at Figure 2. - Figure 10. What is the difference between Cosine Similarity and Sim. Aren't they the same as defined in Equation 3? Why using different terminology?

Questions

- Author claim that the alignment between Image <-> Text tokens helps the model performing well on VQA tasks and hallucinations tasks. It would be interesting to know if this is a cause (i.e performance on vision tasks is high thanks to that alignment), or a consequence (i.e alignment is high along performance on vision tasks thanks to the vision-language training). A way to verify that could be to compare alignment among different off-the-shelf models (e.g. LLaVA-1.5, LLaVA-1.6, Qwen-VL-Chat, Phi3-Vision) and see if there are differences in their respective alignment scores. With such population of models and scores, it could be possible to rely on statistical testing to establish it. Do you see a covariation between alignment and performance among that population? Can you control the alignment independently of the training on vision-language itself? E.g. manipulating directly P and T such as the similarity is higher/lower, and still observe that covariation between the alignment and performance variables in that population? - As well, the authors claim that an LLM can be seen as a residual stream with refinement blocks acting as steering blocks, and that this architecture is what helps the LLM to implicitly having an alignment between text and image tokens. It seems hard to support this claim without contrasting their experiments with another architecture aligning image and text. For instance, using a simple pre-trained CNN to process the image, and a simple pre-trained RNN (e.g. an LSTM or GRU) to process the text input, and then concatenating/pooling the outputs with a simple connector (e.g c-abstractor or just a plain linear projection) could be used to contrast their findings. Is that something planned by the authors?

Rating

6

Confidence

4

Soundness

3

Presentation

1

Contribution

4

Limitations

- One important limitation is the presentation of this work. The majority of the figures are pixelated and not readable. This is often a minor inconvenience in other works, but in this paper all the results are presented through figures. Figure 5 for instance should be removed or importantly reworked. Lacking of space in a limited-page paper is understandable, but not using TKIZ or vectorized figures is not. This is hurting the overall understanding of an interesting work trying to address interesting questions. This work should not be published as is. - Authors claim that the alignment between Image <-> Text tokens is what helps the model performing well on VQA tasks and hallucinations tasks. Something that is not tested is in which extend this is a cause (i.e performance on vision tasks is high thanks to that alignment), or a consequence (i.e alignment is high along performance on vision tasks thanks to the vision-language training). See "Questions" section of this review. - Authors claim that an LLM can be seen as a residual stream with refinement blocks acting as steering blocks, and that this architecture is what helps the LLM to implicitly having an alignment between text and image tokens. It seems hard to support this claim without contrasting the work with another architecture. See "Questions" section of this review.

Summary

The paper explores how frozen Large Language Models generalize to multimodal inputs without the need for extensive re-training. It introduces the concept of Implicit Multimodal Alignment, which suggests that despite the distinct representations of perceptual and textual tokens, there exists a form of implicit alignment facilitated by the architecture of LLMs. The study leverages experimental setups across single-task and multitask environments to validate the IMA phenomenon and discusses its implications for computational efficiency and model performance.

Strengths

1. The analysis spans various setups and modalities, providing a comprehensive look at how LLMs process multimodal inputs. 2. This study can quickly extend large semantic models to the multimodal domain, effectively reducing computational costs and enhancing the generalizability of LLMs. 3. The results are meaningful, proving the effectiveness of the method.

Weaknesses

1. The concept of alignment within neural networks, although well-explored here, does not offer a groundbreaking methodological advance. The novelty lies more in the application context rather than in the development of new techniques or models. 2. The generalizability of the results is restricted to a subset of model architectures and sizes, potentially limiting the broader applicability of the findings. 3. The visualization experiments are not clear. Current figures are somewhat generic and do not adequately convey the unique aspects of the IMA effect.

Questions

1. How does the IMA effect influence the overall performance of LLMs on standard multimodal tasks? Are there performance trade-offs associated with the alignments observed? 2. Are there indications that these findings could be applicable to other types of models or architectures not covered in the study?

Rating

5

Confidence

2

Soundness

2

Presentation

2

Contribution

2

Limitations

1. The paper lacks implementation details. 2. The study focuses primarily on a specific range of LLMs. More domains and tasks should be included.

Summary

This paper conducts an in-depth study on the generalization capabilities of LLM when handling multimodal inputs without multimodal fine-tuning. It reveals the implicit multimodal alignment (IMA) effect between perceptual and textual tokens within LLMs and finds that this effect is closely related to the model architecture. The IMA effect contributes to enhanced task performance and reduced inference costs. Additionally, the paper proposes methods for model compression and reducing computational overhead, providing valuable insights for the future design and optimization of multimodal models.

Strengths

Conducted an EXTENSIVE series of experiments to validate four hypotheses, providing insightful understanding of the mechanisms on how the MLLMs perceive multimodal information. Based on the four findings, the paper offers relevant implications that explain the effects of IMA on tasks and hallucinations. Proposes a novel approach to model compression by retaining a subnetwork. Experiments are comprehensive and were conducted on various large language models, including OPT, Llama, and Vicuna.

Weaknesses

The textual descriptions are somewhat difficult to understand, with limited explanation of the figures. Insufficient explanation of the subnet part. It is unclear whether the similar performance after 50% sparsity is due to the effectiveness of the WANDA method or the extraction of the subnet.

Questions

I do not fully understand line 247. How is the degree of hallucination measured? What is the relationship between cosine similarity and this measurement? Out of personal curiosity, how long did these experiments take?

Rating

9

Confidence

4

Soundness

4

Presentation

3

Contribution

4

Limitations

See weakness and question

Summary

This work aims to understand multi-modality representation within MLLMs. It provides some interesting findings about how LLMs generalize to non-textual tokens and what helps LLMs to generalize to multimodal tokens. Additionally, several implications are proposed based on these findings.

Strengths

1. The authors present many interesting findings about how LLMs generalize to non-textual tokens. These findings could help in understanding MLLMs and inspire future research. 2. Based on these findings, the authors propose several implications on performance, safety and efficiency. 3. The figures are well-illustrated and effectively support the findings.

Weaknesses

**About experiments on α-SubNet.** --Figure 12 appears incomplete. What does "Avg" mean in this table? --The comparison between task-specific pruning approaches (Wanda) and the proposed task-agnostic approach is not comprehensive. For instance, what is the performance of Wanda sub-network pruning on COCO across other multimodal tasks? In other words, I'm curious about the generalization of the task-specific methods. Including these results would help in understanding the significance of task and modality-agnostic pruning methods. --Additionally, the results in Table 1 of the Appendix should be incorporated into the main paper for better readability. --Overall, I recommend reorganizing the experiments related to α-SubNet. **Others**. --line 1192 is unfinished.

Questions

In the single-task (ST) setting (Fig. 2), the authors use unimodal encoders without text-aligned. Figure 2 illustrates "Multimodal cones: different narrow cones for different modalities." However, I wonder if this phenomenon is related to the unimodal encoders themselves—i.e., different modality features from unimodal encoders might have very different feature distributions, leading to different narrow cones in LLMs. Could the authors conduct new experiments using aligned encoders, such as ImageBind [1], as different modality encoders? This would help determine whether the different narrow cones in LLMs are due to different modalities or different encoders. Ref: 1. ImageBind: One Embedding Space To Bind Them All. CVPR 2023

Rating

7

Confidence

4

Soundness

4

Presentation

4

Contribution

4

Limitations

As stated by the authors, the generalization of the findings, to larger and more powerful models, with different architectures, including proprietary ones remains to be seen