Convolution Neural Networks (CNN) and Recurrent Neural Networks (RNN), which are the main research methods of image caption, have developed rapidly. Nevertheless lacking of global consciousness in image caption has not been completely solved. Separation from the bottom-up visual attention mechanism and the top-down visual attention mechanism has been widely used in image description and visual question and answers. In this article, we put forward image description based on a global attention mechanism research methods. The global attention prior channel is added to the infrastructure to extract the global information features while learning the local features. The attention to the object and other outstanding level of image region is calculated, and the global image features are enhanced.
Paper
Full text
Research for image caption based on global attention mechanism
OpenAlex · Multimodal Machine Learning Applications · 2020
Abstract
Convolution Neural Networks (CNN) and Recurrent Neural Networks (RNN), which are the main research methods of image caption, have developed rapidly. Nevertheless lacking of global consciousness in image caption has not been completely solved. Separation from the bottom-up visual attention mechanism and the top-down visual attention mechanism has been widely used in image description and visual question and answers. In this article, we put forward image description based on a global attention mechanism research methods. The global attention prior channel is added to the infrastructure to extract the global information features while learning the local features. The attention to the object and other outstanding level of image region is calculated, and the global image features are enhanced.