Summary
While previous studies on vision transformers focused on how self-attention groups relevant tokens, this paper analyzes how self-attention contextualizes tokens to understand comprehensive inter-token relationships across the entire image. To this end, this paper proposes using the Singular Value Decomposition (SVD) to analyze the query-key interaction $\textbf{W}^{\top}_q \textbf{W}_k$. Each singular vector of the query-key projection layers can be interpreted as capturing a certain type of visual semantic information. Thus, similarities between left and right singular vectors can reveal how different visual semantics interact with each other. Through extensive analysis, this paper concludes that vision transformers tend to first attend to similar tokens to form local visual semantics, and then attend to dissimilar tokens to capture global contexts of the image.
Strengths
S1. **Writing**: The paper is well written and clearly motivated.
S2. **Novelty**: The proposed analysis via SVD is innovative.
S3. **Technical soundness**: The justification for using SVD is well-founded theoretically and supported by solid quantitative and qualitative empirical analysis.
S4. **Potential for broader application**: Despite being applied to image analysis, the proposed technique is generic and could potentially be applied to various domains, including video and audio understanding..
Weaknesses
W1. **Familiar conclusion**: The general finding of this paper, i.e., group early and contextualize later, is already well known from many earlier papers [a,b]. This paper only reconfirms the same conclusion in an explainable manner.
W2. **Limited analysis scope**: This paper lacks analysis on different training objectives such as masked image modeling (MIM) [c,d] or masked feature prediction (JEP) [e,f]. Previous literatures [a,b] show that training objectives determine where the learner focuses; ViT with supervised training or instance discrimination self-supervision (SimCLR or MoCo) act similar as what this paper found, but ViTs trained with MIM shows that the learner still focuses on local tokens in a deeper layer. The observations in L6-8, 45-47, 138-141 might only be correct for certain training objectives.
Justification of rating: \
Despite not deriving a novel conclusion and lacking analysis on other training objectives, the proposed SVD-based analysis technique is novel, provides interpretability on query-key interactions, and is technically sound. It has potential applications in various domains.
[a] Xie et al., “Revealing the Dark Secrets of Masked Image Modeling,” CVPR, 2023.\
[b] Shekhar et al., “Objectives Matter: Understanding the Impact of Self-Supervised Objectives on Vision Transformer Representations,” ICLRW, 2023.\
[c] Xie et al., “SimMIM: A Simple Framework for Masked Image Modeling,” CVPR, 2022.\
[d] He et al., “Masked Autoencoders Are Scalable Vision Learners,” CVPR, 2022.\
[e] Assra et al., “Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture,” CVPR, 2023.\
[f] Baevski et al., “data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language,” ICML, 2022.
Questions
Q1. In Fig. 3 & S3, there are slight increases in weighted cosine similarity in the final layers for all models except for the original ViTs. Does this imply that ViTs group similar tokens again in the deepest layers? Please elaborate on why this happens and what information ViTs capture in these layers.
Q2. For Fig. 4, visualizing attention maps of singular modes with fixed input images would help readers understand how ViTs adapt their focus according to the layer.
Q3. How does the diversity of visual information captured by singular vectors change across layers? In other words, how does the redundancy between singular vectors vary?