Scholarlink

<p>Most Speech Emotion Recognition (SER) systems assume complete and well-structured speech input; however, real-world speech is often incomplete, irregular, and affected by disfluencies or pronunciation variability and temporal inconsistency. SER systems usually assume complete, well-formed input. A hybrid deep learning architecture is proposed that combines convolutional neural networks (CNNs), bidirectional long short-term memory (BiLSTM) and transformer models. It has been trained to recognise emotions in fragmented, unstructured speech. We use Mel-Frequency Cepstral Coefficients (MFCCs) as acoustic features and introduce a new Partial-Disordered Speech Augmentation Strategy (PDSAS) to mimic PDSAS-simulated conditions of impaired speech, such as temporal cropping, frame masking, frame shuffling, and signal dropout. The architecture is hierarchical. The CNN captures local spectral features, the BiLSTM models bidirectional temporal dependencies, and the Transformer enhances global contextual learning through attention mechanisms. A fivefold cross-validation strategy is used during training and testing to ensure that the model is strong and can work well with new data. Results from experiments on the speaker-independent CREMA-D dataset show that the proposed model always does better than CNN-only, BiLSTM-only, and Transformer-only baselines in terms of accuracy and F1-score. These results show that the proposed framework works well for recognising speech emotions in real life, especially when the speech comes from speakers under simulated degraded speech conditions.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A hybrid CNN–BiLSTM–transformer fusion model for emotion recognition from partial and disordered speech

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年6月22日

查看全文
  • Siva Ramakrishna Tatapudi,
  • G Jaya Suma

摘要

Most Speech Emotion Recognition (SER) systems assume complete and well-structured speech input; however, real-world speech is often incomplete, irregular, and affected by disfluencies or pronunciation variability and temporal inconsistency. SER systems usually assume complete, well-formed input. A hybrid deep learning architecture is proposed that combines convolutional neural networks (CNNs), bidirectional long short-term memory (BiLSTM) and transformer models. It has been trained to recognise emotions in fragmented, unstructured speech. We use Mel-Frequency Cepstral Coefficients (MFCCs) as acoustic features and introduce a new Partial-Disordered Speech Augmentation Strategy (PDSAS) to mimic PDSAS-simulated conditions of impaired speech, such as temporal cropping, frame masking, frame shuffling, and signal dropout. The architecture is hierarchical. The CNN captures local spectral features, the BiLSTM models bidirectional temporal dependencies, and the Transformer enhances global contextual learning through attention mechanisms. A fivefold cross-validation strategy is used during training and testing to ensure that the model is strong and can work well with new data. Results from experiments on the speaker-independent CREMA-D dataset show that the proposed model always does better than CNN-only, BiLSTM-only, and Transformer-only baselines in terms of accuracy and F1-score. These results show that the proposed framework works well for recognising speech emotions in real life, especially when the speech comes from speakers under simulated degraded speech conditions.