Scholarlink

<p>Emotion recognition is a necessary yet challenging task, which is becoming increasingly popular in the applications of human-computer interaction systems. Effectively utilizing emotion information and fusing the interactive information existing in different modalities is still a challenging problem. To address this issue, we propose a novel model for multimodal speech emotion recognition using cross-modal convolution attention and multi-acoustic feature fusion. We introduces cross-modal convolutional attention module (CMCA) and multi-acoustic feature fusion module (MAFF). First, MAFF is proposed to fuse multiple acoustic features to enhance the emotion expression of speech modality, utilizing the complementarity of acoustic features. Then, CMCA is proposed to fuse speech and text features which helps to enhance the interaction between speech and text sequences, thereby improving the accuracy of emotion recognition. In addition, we use speaker-independent and speaker-dependent five-fold cross-validation strategies to validate the effectiveness of our model. Experimental results based on the IEMOCAP dataset show that our model outperforms multiple state-of-the-art models.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Multimodal emotion recognition using cross-modal convolutional attention and multi-acoustic feature fusion

  • 期刊论文

  • 发表日期：

    2026年6月2日

查看全文
  • Yanxiang Chen,
  • Wenchao Jiang,
  • Jianing Zhang,
  • Hong Xiao,
  • Zhiming Zhao,
  • Tao Wu

摘要

Emotion recognition is a necessary yet challenging task, which is becoming increasingly popular in the applications of human-computer interaction systems. Effectively utilizing emotion information and fusing the interactive information existing in different modalities is still a challenging problem. To address this issue, we propose a novel model for multimodal speech emotion recognition using cross-modal convolution attention and multi-acoustic feature fusion. We introduces cross-modal convolutional attention module (CMCA) and multi-acoustic feature fusion module (MAFF). First, MAFF is proposed to fuse multiple acoustic features to enhance the emotion expression of speech modality, utilizing the complementarity of acoustic features. Then, CMCA is proposed to fuse speech and text features which helps to enhance the interaction between speech and text sequences, thereby improving the accuracy of emotion recognition. In addition, we use speaker-independent and speaker-dependent five-fold cross-validation strategies to validate the effectiveness of our model. Experimental results based on the IEMOCAP dataset show that our model outperforms multiple state-of-the-art models.