Text-enhanced cross-modal reinforced fusion network for multimodal sentiment analysis
摘要
Multimodal sentiment analysis (MSA) aims to infer affect from text, audio, and visual signals. Although recent fusion models achieve strong accuracy, many still suffer from two problems: spurious lexical correlations in the text stream and unstable fusion under noisy non-textual evidence. We propose the Text-enhanced Cross-modal Reinforced Fusion Network (TECRFN), a unified framework for unaligned MSA that addresses both issues. TECRFN first employs a Text Debiasing Module (TDM) to suppress sentiment-irrelevant lexical shortcuts and produce a cleaner textual anchor. It then uses a Text-enhanced Transformer (TET) and a Text-guided Cross-modal Reinforced Transformer (TGCRT) to inject textual semantics into audio and vision, perform query-adaptive dual-path fusion, and recalibrate noisy channels through similarity-biased gating and channel attention. A memory encoder and a unimodal label generation module further preserve long-range context and modality-specific discriminability. Experiments on CMU-MOSI and CMU-MOSEI show that TECRFN achieves state-of-the-art or highly competitive performance across multiple metrics. Additional ablation, robustness, and sensitivity studies demonstrate that the proposed design improves alignment quality, stability, and resistance to bias shift.