Multimodal sentiment analysis based on noise injection and adaptive multi-view dynamic fusion
摘要
Multimodal sentiment analysis (MSA) infers human sentiment by jointly modeling textual, auditory, and visual signals and is a key component in intelligent interaction systems. Performance, however, is often degraded by cross-modal noise and insufficiently optimized feature fusion. This study proposes NIAMDF, a model based on noise injection and adaptive multi-view dynamic fusion. To suppress the propagation of cross-modal noise, an adversarial noise suppression module couples noise injection with adversarial training, enabling intervention during feature generation. A collaborative constraint mechanism is additionally imposed, promoting consistency between disentangled representations and the original semantics. For adaptive fine-grained fusion, a multi-view design is adopted: modality-shared dynamic knowledge fusion performs context-adaptive filtering of shared representations, modality-specific cross-modal attention extracts modality-unique affective cues, and multi-expert gating fusion aggregates multi-perspective representations while dynamically adjusting modality contribution weights. Experiments are conducted on three widely used MSA datasets (CMU-MOSI, CMU-MOSEI, and CH-SIMS). NIAMDF achieves competitive performance relative to state-of-the-art models. The model also remains effective in challenging cases involving inconsistent affective expressions across modalities, cross-modal noise interference, and information redundancy.