Prompt-Guided Mixture-of-Experts for Robust Multimodal Sentiment Analysis with Missing Modalities

In multimodal emotion recognition tasks, the widespread issue of missing modalities severely hinders model performance and generalization ability. To address this challenge, we propose PMoE, a Prompt-guided Mixture-of-Experts framework for robust multi-modal emotion recognition. Built upon a frozen, pretrained Transformer backbone, PMoE introduces a missing modality generation scheme that combines generative prompts and confidence-weighted fusion, effectively enhancing the quality of missing information compensation. A two-stage dynamic routing mechanism is further employed within the MoE layer to enable more flexible cross-modal feature fusion. In addition, a self-distillation strategy is adopted to stabilize training and improve generalization by leveraging historical model outputs as soft targets for progressive optimization. Experimental results on four public datasets—CMU-MOSI, CMU-MOSEI, IEMOCAP, and CH-SIMS—demonstrate that PMoE consistently outperforms existing baselines, especially under conditions of severe modality incompleteness, validating the effectiveness and robustness of the proposed framework.