Scholarlink

<p>With the widespread application of intelligent dialogue systems, Multimodal emotion recognition in conversations has gained increasing attention. Emotional information in dialogues is often expressed through multiple modalities, including text, audio, and vision cues. However, existing methods generally face difficulties in semantic alignment when fusing heterogeneous modal features, and many adopt single-granularity modeling approaches that ignore the multi-level semantic structure of conversations, limiting the accuracy and generalization capabilities of emotion recognition. To address these issues, this paper proposes an Auxiliary Enhanced Adaptive Entropy-guided Multi-Granularity Network (AE<sup>2</sup>MGNet). The model generates semantically enhanced auxiliary modality through Low-rank Cosine Auxiliary Modality Modeling, effectively alleviating feature distribution differences between heterogeneous modalities. Simultaneously, it constructs Multi-granularity heterogeneous graph structures at sentence, turn, and dialogue levels, inputting the auxiliary modality along with the original three modalities as nodes to achieve hierarchical emotion modeling from local details to global context. Additionally, it introduces an adaptive entropy weighting mechanism to dynamically adjust the contribution of different fusion strategies, enabling the model to intelligently integrate multi-level features. Experimental results demonstrate that <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(\textrm{AE}^2\)</EquationSource> </InlineEquation>MGNet achieves accuracies of 73.81%, 68.01%, and 46.49% on the IEMOCAP, MELD, and CMU-MOSEI datasets, respectively, with weighted F1 scores of 73.82%, 67.13%, and 45.10%, significantly outperforming existing methods.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Auxiliary enhanced adaptive entropy-guided multi-granularity networks for multimodal emotion recognition in conversations

  • 期刊论文

  • 发表日期：

    2026年3月9日

查看全文
  • Jun Wu,
  • Yu Chen,
  • Panpan Chen,
  • Shuai Guo,
  • Jiahui Huang,
  • Xinyi Zhu,
  • Qun Zhang

摘要

With the widespread application of intelligent dialogue systems, Multimodal emotion recognition in conversations has gained increasing attention. Emotional information in dialogues is often expressed through multiple modalities, including text, audio, and vision cues. However, existing methods generally face difficulties in semantic alignment when fusing heterogeneous modal features, and many adopt single-granularity modeling approaches that ignore the multi-level semantic structure of conversations, limiting the accuracy and generalization capabilities of emotion recognition. To address these issues, this paper proposes an Auxiliary Enhanced Adaptive Entropy-guided Multi-Granularity Network (AE2MGNet). The model generates semantically enhanced auxiliary modality through Low-rank Cosine Auxiliary Modality Modeling, effectively alleviating feature distribution differences between heterogeneous modalities. Simultaneously, it constructs Multi-granularity heterogeneous graph structures at sentence, turn, and dialogue levels, inputting the auxiliary modality along with the original three modalities as nodes to achieve hierarchical emotion modeling from local details to global context. Additionally, it introduces an adaptive entropy weighting mechanism to dynamically adjust the contribution of different fusion strategies, enabling the model to intelligently integrate multi-level features. Experimental results demonstrate that \(\textrm{AE}^2\) MGNet achieves accuracies of 73.81%, 68.01%, and 46.49% on the IEMOCAP, MELD, and CMU-MOSEI datasets, respectively, with weighted F1 scores of 73.82%, 67.13%, and 45.10%, significantly outperforming existing methods.