A Cloud-Based Cross-Modal Transformer for Emotion Recognition and Adaptive Human-Computer Interaction

Emotion recognition is a fundamental component of next-generation human-computer interaction (HCI), enabling machines to perceive, understand, and respond to users’affective states. However, existing systems often rely on single-modality analysis-such as facial expressions, speech tone, or textual sentiment-resulting in limited robustness and poor generalization in real-world environments. To address these challenges, this study proposes a Cloud-Based Cross-Modal Transformer (CMT) framework for multimodal emotion recognition and adaptive human-computer interaction. The proposed model integrates visual, auditory, and textual signals using pretrained encoders-Vision Transformer (ViT), Wav2Vec2, and BERT-and employs a cross-modal attention mechanism to capture complex interdependencies among heterogeneous features. By leveraging cloud computing infrastructure with distributed training (Kubernetes and TensorFlow Serving), the system enables scalable, low-latency emotion recognition for large-scale user interactions. Experiments conducted on benchmark datasets including IEMOCAP, MELD, and AffectNet demonstrate that the proposed Cloud-Based Cross-Modal Transformer (CMT) achieves state-of-the-art performance, improving the F1-score by 3.0% and reducing cross-entropy loss by 12.9% compared to the best fusion-based baselines. Additionally, cloud deployment evaluations show an average response latency of 128 ms, representing a 35% reduction compared with conventional transformer-based fusion systems. These results confirm that the proposed CMT framework enables efficient, real-time emotion recognition and adaptive feedback in interactive applications such as intelligent customer service, virtual tutoring, and affective computing interfaces.These results highlight the effectiveness and scalability of the proposed CMT framework, marking an important step toward cloud-native affective computing and emotionally intelligent interactive systems.