Speech Emotion Recognition (SER) is designed to automatically detect human emotions from speech, a task that is essential for improving human-computer interaction. Motivated by the need to enrich feature representation, we unveil a new SER framework that integrates wav2vec2 embeddings obtained from raw speech waveforms, with a hybrid CNN-BiLSTM architecture and an attention pooling mechanism to capture both local and long-range emotional patterns. To further enrich emotional representation, Valence-Arousal-Dominance (VAD) features are incorporated using a dedicated MLP module, addressing the challenge of distinguishing acoustically similar emotions. The final classification is performed using a MLP that integrates both acoustic and emotional embeddings. Our approach is evaluated on the IEMOCAP corpus across four key emotion categories: Angry, Happy, Sad, and Neutral. The results obtained from the ablation study indicate that the proposed approach performed better than the existing state-of-the-art techniques with an improvement in accuracy of 80.47%. Furthermore, it obtained an unweighted accuracy (UAR) of 81.54%, indicating strong performance across all emotion classes. The findings highlight the effectiveness of integrating pretrained audio representations with emotion-aware features and demonstrate robustness against class imbalance through selective data augmentation.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Emotion Recognition from Speech via Hybrid CNN-BiLSTM with Attention Pooling and VAD Fusion

  • S. Sreeja,
  • Philomina Simon

摘要

Speech Emotion Recognition (SER) is designed to automatically detect human emotions from speech, a task that is essential for improving human-computer interaction. Motivated by the need to enrich feature representation, we unveil a new SER framework that integrates wav2vec2 embeddings obtained from raw speech waveforms, with a hybrid CNN-BiLSTM architecture and an attention pooling mechanism to capture both local and long-range emotional patterns. To further enrich emotional representation, Valence-Arousal-Dominance (VAD) features are incorporated using a dedicated MLP module, addressing the challenge of distinguishing acoustically similar emotions. The final classification is performed using a MLP that integrates both acoustic and emotional embeddings. Our approach is evaluated on the IEMOCAP corpus across four key emotion categories: Angry, Happy, Sad, and Neutral. The results obtained from the ablation study indicate that the proposed approach performed better than the existing state-of-the-art techniques with an improvement in accuracy of 80.47%. Furthermore, it obtained an unweighted accuracy (UAR) of 81.54%, indicating strong performance across all emotion classes. The findings highlight the effectiveness of integrating pretrained audio representations with emotion-aware features and demonstrate robustness against class imbalance through selective data augmentation.