Aligned Information Bottleneck for Multimodal Social Sentiment Analysis

Multimodal sentiment analysis (MSA) infers affect from synchronized text, speech, and visual signals, but it still faces cross-modal misalignment and noisy fusion, especially in naturalistic social communication. Differences in temporal resolution, semantic granularity, and modality-specific noise mean that simple concatenation or coarse interaction can amplify redundant cues from one modality while weakening informative signals from others. Recent approaches based on the Information Bottleneck (IB) principle help reduce redundancy. However, matrix-based Rényi-entropy estimators typically rely on a fixed low-rank truncation, making the resulting bottleneck sensitive to spectrum variation across batches, modalities, and difficulty levels. We propose a text-anchored IB framework for MSA that jointly addresses alignment and fusion. First, we construct a language-centered semantic space using a pretrained Qwen-3 encoder and map acoustic and visual features into this space, yielding an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">aligned information bottleneck representation</i> that reduces cross-modal semantic gaps before fusion. Second, we introduce an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">energy-adaptive low-rank Rényi-entropy based information bottleneck</i>. It builds on matrix-based Rényi-entropy estimators and selects the effective rank via an energy criterion. This suppresses redundant directions while preserving label-relevant structure. We evaluate the proposed framework on standard utterance-level MSA benchmarks with text, audio, and video. Extensive experiments and ablation studies show that our method improves over strong baselines across standard evaluation metrics. We also observe better robustness under modality noise and imbalance, supporting the value of text-anchored alignment and energy-adaptive low-rank IB for multimodal affect modeling.