Df-emii: a dual-level multimodal fusion framework for sentiment analysis and emotion recognition with applications in public opinion monitoring
摘要
The growing influence of social media has transformed online public opinion into a complex multimodal environment, where textual expressions, vocal cues, and visual behaviors jointly convey rich affective information. Accurate multimodal sentiment analysis and emotion recognition are therefore essential for understanding public opinion dynamics and emotionally driven interactions on online platforms. However, existing multimodal models often rely on shallow fusion strategies and limited global reasoning, which can lead to incomplete emotional representations and unstable cross-modal interactions. To address these challenges, we propose DF-EMII, a dual-level multimodal fusion framework for sentiment analysis and emotion recognition in public opinion scenarios. DF-EMII integrates three complementary components: (1) ENSA, which enhances intra-modal representations through neuron-level saliency calibration and group-aware attention; (2) DACF, a two-stage adaptive fusion mechanism that aligns modalities under textual semantics and performs reciprocal cross-modal refinement; and (3) GMED, which incorporates memory-guided global context to preserve long-range semantic and emotional coherence. Extensive experiments on CMU-MOSI, CMU-MOSEI, and IEMOCAP demonstrate that DF-EMII consistently outperforms state-of-the-art approaches on both sentiment regression and emotion classification tasks. These results indicate that DF-EMII effectively captures subtle affective cues across modalities, providing a robust computational framework for multimodal sentiment analysis and emotion understanding in public opinion monitoring applications.