Graph Dynamic Fusion Network with Contrastive Learning for Multimodal Emotion Recognition in Conversation
摘要
Multimodal Emotion Recognition in Conversations (MERC) aims to detect emotional states through the integration of multimodal signals in conversational video data. Current methods commonly fuse multimodal information by modeling intra-modal and cross-modal interactions within graph-based architectures. Nevertheless, these approaches often accumulate redundant information across network layers. In addition, heterogeneity and misalignment among multimodal features tend to introduce interfering noise. To tackle these challenges, this paper proposes a Graph Dynamic Fusion Network with Contrastive Learning (GDFNCL), designed to effectively harness complementary relationships across modalities in conversational settings. Specifically, the method constructs a dynamic graph network to adaptively model interactions between multimodal representations and capture cross-modal dependencies through graph structural learning. In parallel, contrastive learning is applied to attract feature representations of similar samples while repelling those of dissimilar ones. This integrated strategy effectively alleviates issues of modality heterogeneity and misalignment, while minimizing redundant information, leading to substantial gains in recognition accuracy and overall performance. Experiments conducted on two benchmark datasets (IEMOCAP and MELD) confirm that the proposed GDFNCL framework significantly surpasses existing state-of-the-art methods in multimodal emotion recognition.