A Systematic Approach on Attention Based Multimodal Sentiment Analysis in Convolutional Neural Network using Deep Learning Techniques

Sentiment analysis represents an emerging field of study in both natural language processing and computational intelligence with consumer evaluations of products and services are advancing in autonomy. Multimodal sentiment analysis aims to determine a user’s sentiment using multimodal data. It seeks to describe a novel approach to sentiment categorization that employs deep learning and natural language processing. The proposed framework for attention-based multimodal sentiment classification combines convolutional neural networks for image data with bidirectional encoder representations from transformers for textual data. The synergy between these two modalities enhances the model’s capability of capturing abundant contextual information and visual signals. The framework employs a convolutional neural network for image data that recovers hierarchical features while preserving spatial relationships. Simultaneously, the BERT model encodes textual information, enabling the model to comprehend semantic sequence. Combining these modalities requires an attention mechanism to learn and dynamically evaluate the importance of visual and textual signals during classification. Attention-based fusion enhances sentiment categorization performance by focusing on the most informative elements from both modalities. Extensive evaluations utilizing the publicly available CMU Multimodal Opinion Sentiment and Emotion Intensity dataset demonstrate the utility of the proposed framework with an accuracy of 95.87%. The experimental findings indicate that the convolutional neural network with the BERT method captures complex sentiment patterns across various domains more effectively.