Summary
The paper introduces MULTIMODALCAUSALTRACE, an algorithm designed to identify hidden states in large language models (LLMs) that store factual information, specifically extending this capability to vision-language models where images are encoded as visual tokens. Building on previous work in information storage identification, MULTIMODALCAUSALTRACE constructs three models: a clean model, a corrupted model (where visual tokens are randomly replaced, perturbing the hidden states), and a restored model (where some corrupted hidden states are replaced with clean ones). By conducting mediation analysis on the causal relationship between state corruption and next-token prediction outcomes, the algorithm can pinpoint layers associated with different facts. The study reveals that multi-modal LLMs exhibit distinct behaviors in information storage and propagation, such as storing facts in middle layers rather than earlier transformer layers.
Additionally, the paper contributes the VQA-constraints dataset, derived from existing visual QA datasets and annotated with constraints.
The paper also proposes MULTEDIT, an algorithm for inserting or correcting factual knowledge in multi-modal LLMs. MULTEDIT optimizes the projection matrix in the MLP layers of transformers to minimize the mean squared difference between the projection output and the value vector that maximizes the prediction probability of the desired output. Empirical results demonstrate that MULTEDIT effectively corrects factual errors in VQA tasks.
Strengths
+ The paper tackles an important task of identifying and correcting factual information in visual language model, a topic that received relatively less attention than in language modeling.
+ The paper provided strong motivation to the problem and background it addresses, as well as clear visual and formal presentation of the methodology adapted.
+ The proposed solution is comprehensive and encompassed various aspect of the information storage analysis problem. It delivers valuable insights to some practical challenges of adapting existing LLM-based methods to vision-language models such as the increased number of constraint tokens due to visual encoding.
Weaknesses
+ The uniqueness and impact of multi-modal models are not thoroughly demonstrated. As stated in the paper, the proposed methods is largely based on the canonical Causal Trace(CT) and Rank-One Model Editing (ROME), with modifications in implementation allowing them to work for vision-language models. However, the choice (or absence) of most of such modifications are not well-motivated and no ablation study provided to understand their exact contribution. For example, the proposed MULTEDIT largely resembles the existing ROME method, with differences in how the key vector is found and how the optimal value vector is acquired. But it is not clear from the text why such changes are necessary and/or make the method works better for visual language model.
+ The lack of discussion on multi-modal properties hinders the paper's novelty. The overall structure of the paper is similar to [23], and the main text does not substantially differentiate the methodology from prior work.
+ Minor formatting issue in the reference: journal/conference names are missing in some items (such as [23] [24])
(The reference indices of the original submission are used throughout this review)
Questions
+ In vanilla ROME editing, the value vector is found by optimizing both the probability of the desired output and similarity to the essence of the original sentence. But in MULTEDIT the later term is dropped. What is the purpose of such change and how does the updated objective preserve the understanding of the overall prompt?
+ In the creation of VQA-Constraints, does the authors manually correct all annotations generated by GPT-4? If so, what is the main advantage of starting with automated annotations?
Limitations
The authors included a limitation section in the appendix covering drawbacks of the proposed method and potential negative societal impacts.