The majority of existing methods for Visual Question Answering are primarily based on Recurrent Neural Networks(RNNs) with attention to extract question features and outputs the last hidden state for modal fusion. However, only the last hidden state cannot preserve precise location information, which may lead to semantic confusion. Our work adopts Multiple Encoder Block to replace RNNs, and propose a Double Attention Module which includes Objective Attention and Spatial Attention to acquire more important regions or objects. The proposed method is evaluated on VQA 2.0 and CLEVR, and obtain competitive result.
Paper
Full text
MDAnet: Multiple Fusion Network with Double Attention for Visual Question Answering
Semantic Scholar · Computer Science · 2019
Abstract
The majority of existing methods for Visual Question Answering are primarily based on Recurrent Neural Networks(RNNs) with attention to extract question features and outputs the last hidden state for modal fusion. However, only the last hidden state cannot preserve precise location information, which may lead to semantic confusion. Our work adopts Multiple Encoder Block to replace RNNs, and propose a Double Attention Module which includes Objective Attention and Spatial Attention to acquire more important regions or objects. The proposed method is evaluated on VQA 2.0 and CLEVR, and obtain competitive result.