Enhancing speech-based emotion recognition with transformer models and attention mechanisms
摘要
Speech emotion recognition (SER) is one of the important tasks. It is widely used in human-computer interaction, health care, customer satisfaction, and emotionally based education systems. Conventional techniques for speech-based emotion recognition frequently need help to identify the subtle and complicated characteristics included in vocal expressions. Extracting the features relevant to emotion is one of the difficult challenges in existing methods. This research employed transformer models with multi-head self-attention as novel methods to improve the effectiveness of speech-based emotion recognition. Important features are extracted from speech datasets, namely chroma features, mel spectrograms, and Mel-Frequency Cepstral Coefficients (MFCCs). Unlike prior Transformer applications to SER that rely on raw waveforms or single feature types, the proposed model introduces an attention-guided fused feature embedding mechanism that uniquely combines fused spectral-harmonic feature representation with Multi-head attention methods, which were utilized to selectively emphasize the emotionally relevant features from the dataset and improve classification accuracy by extracting relevant portions of input data. The CREMA dataset was utilized to evaluate the proposed work’s performance by 5-fold and 10-fold cross-validation. As a result of achieving an accuracy of 87.4%, more effective and robust emotion detection algorithms can be developed. To calculate the effectiveness of the proposed work, our algorithm is compared with previously existing methods, including CNN-LSTM (74.1%), BiGRU+WavVec (79.6%), and Co-Attention fusion networks (82.5%). Ablation studies confirm that the research model shows that attention mechanisms are critical for understanding and interpreting emotional signals in speech. This work lays the groundwork for further advancements in the field by highlighting the efficacy of Transformer models in recognizing emotions.