Scholarlink

<p>To decode human emotions from speech, intelligent systems require Speech Emotion Recognition (SER). However, it remains challenging to assess SER in multilingual and multi-regional contexts because language and culture are not uniform, and acoustics vary. Among these barriers is the fact that traditional models do not combine acoustic features with semantic context, preventing them from generalizing to languages with limited resources and multifaceted expressions of emotion. It is recommended that the challenge be addressed by proposing a robust multilingual SER framework that integrates a new threshold-based feature selection mechanism with contextual fusion based on Large Language Models (LLMs). The feature selection procedure yields an Otsu-Human Memory Optimizer (Otsu-HMO) that dynamically selects discriminative and emotion-based features by combining inter-class variance-based thresholding with memory-based reinforcement plans. The semantic gap is closed by a Context-Learning Emotion Fusion with Cross-Attention (CLEF-XA) module, which is a cross-attention mechanism based on XLM-RoBERTa embeddings that enables fine-tuning of the acoustic representation to the language context. Finally, a Deep Convolutional Residual Attention-based Deterministic Uncertainty (DCRA-DU) classifier, trained on the Battlefield Optimizer (BMO), provides predictive stability for emotion recognition, uncertainty assessment, and adversarial multilingual regularization. The experimental findings on the EMO-DB (German), BanglaSER (Bengali), and ASVP-ESD (multilingual) data sets indicate that the proposed system achieves 95.01, 96.00, and 94.00 accuracies with 4.2, 3.8, and 5.1 percentage points improvement in precision, recall, and macro F1-score, respectively, over the available methods. The framework provides a cross-culturally inclusive service interface, with significant potential for multilingual virtual assistants, emotion-based artificial intelligence, and cross-cultural human–computer interaction.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A Cross-Residual Attention and Uncertainty-Aware Framework for Multilingual Speech Emotion Recognition

  • 期刊论文

  • 发表日期：

    2026年3月15日

查看全文
  • Deepak Kumar Yadav,
  • Ati Jain,
  • Lalji Prasad,
  • Rashmi Yadav

摘要

To decode human emotions from speech, intelligent systems require Speech Emotion Recognition (SER). However, it remains challenging to assess SER in multilingual and multi-regional contexts because language and culture are not uniform, and acoustics vary. Among these barriers is the fact that traditional models do not combine acoustic features with semantic context, preventing them from generalizing to languages with limited resources and multifaceted expressions of emotion. It is recommended that the challenge be addressed by proposing a robust multilingual SER framework that integrates a new threshold-based feature selection mechanism with contextual fusion based on Large Language Models (LLMs). The feature selection procedure yields an Otsu-Human Memory Optimizer (Otsu-HMO) that dynamically selects discriminative and emotion-based features by combining inter-class variance-based thresholding with memory-based reinforcement plans. The semantic gap is closed by a Context-Learning Emotion Fusion with Cross-Attention (CLEF-XA) module, which is a cross-attention mechanism based on XLM-RoBERTa embeddings that enables fine-tuning of the acoustic representation to the language context. Finally, a Deep Convolutional Residual Attention-based Deterministic Uncertainty (DCRA-DU) classifier, trained on the Battlefield Optimizer (BMO), provides predictive stability for emotion recognition, uncertainty assessment, and adversarial multilingual regularization. The experimental findings on the EMO-DB (German), BanglaSER (Bengali), and ASVP-ESD (multilingual) data sets indicate that the proposed system achieves 95.01, 96.00, and 94.00 accuracies with 4.2, 3.8, and 5.1 percentage points improvement in precision, recall, and macro F1-score, respectively, over the available methods. The framework provides a cross-culturally inclusive service interface, with significant potential for multilingual virtual assistants, emotion-based artificial intelligence, and cross-cultural human–computer interaction.