Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers

Transformers are increasingly dominating multi-modal reasoning tasks, such as\nvisual question answering, achieving state-of-the-art results thanks to their\nability to contextualize information using the self-attention and co-attention\nmechanisms. These attention modules also play a role in other computer vision\ntasks including object detection and image segmentation. Unlike Transformers\nthat only use self-attention, Transformers with co-attention require to\nconsider multiple attention maps in parallel in order to highlight the\ninformation that is relevant to the prediction in the model's input. In this\nwork, we propose the first method to explain prediction by any\nTransformer-based architecture, including bi-modal Transformers and\nTransformers with co-attentions. We provide generic solutions and apply these\nto the three most commonly used of these architectures: (i) pure\nself-attention, (ii) self-attention combined with co-attention, and (iii)\nencoder-decoder attention. We show that our method is superior to all existing\nmethods which are adapted from single modality explainability.\n