IAGF: a dual-path collaborative inconsistency-aware global fusion network for multimodal sarcasm detection
摘要
Multimodal sarcasm detection aims to identify ironic intent from textual and visual information, with the core challenge lying in capturing semantic inconsistencies between different modalities. Existing methods primarily rely on local implicit interactions, such as attention mechanisms or graph-based modelling, to learn cross-modal relationships. Although these methods are capable of extracting latent multimodal cues, they often lack explicit semantic constraints and interpretable inconsistency modelling mechanisms, making it difficult to capture holistic sample-level mismatches between text and images. To address this limitation, this paper proposes IAGF, a dual-path collaborative inconsistency-aware global fusion network. IAGF combines implicit global cross-modal semantic learning with explicit sample-level inconsistency modelling. Specifically, the cross-modal global fusion module performs multi-dimensional subspace transformations in a shared global space to enhance multi-modal representations whilst preserving mono-modal contextual information. Concurrently, the bidirectional inconsistency learning module employs a modal completion network to generate the expected representation of one modality from another, and measures the semantic deviation between this expected representation and the actual representation. Subsequently, the learned discrepancy signal is incorporated into the classification process. Experimental results on MMSD and MMSD2.0 demonstrate that IAGF exhibits consistent and stable performance advantages, outperforming current state-of-the-art methods on MMSD, and validating the importance of bidirectional discrepancy learning for modelling complex outliers.