An essential first step in developing emotionally intelligent systems that can react to users more naturally is the recognition of emotions from speech. Human-computer interaction, healthcare, education, and customer service are just a few of the fields where this capability could enhance user experience. Pitch, tone, and rhythm are some of the components that speech uses to convey emotional information. These cues, when properly interpreted, enable machines to recognise human emotions and respond appropriately. In this paper, we present a deep learning-based method that estimates the strength of an emotion’s expression in speech in addition to identifying its type. The system uses Mel-Frequency Cepstral Coefficients (MFCCs) to first extract audio features in order to accomplish this. After that, an autoencoder compresses these features to create a lower-dimensional representation while preserving the emotional content. Two distinct modules receive this compressed data: one for categorising the emotion category and another for forecasting the emotion’s intensity on a continuous scale. In order to enhance performance and generalisation, the entire system is trained using a multi-task learning framework, which enables it to learn both tasks simultaneously. During training, class weighting and stratified sampling were used to address the dataset’s class imbalance. Furthermore, the suggested model outperformed baseline classifiers like SVM and CNN-LSTM in comparison. Strong results are obtained from our experiments on the RAVDESS dataset. The model’s mean absolute error (MAE) for intensity prediction was 0.1523 and its accuracy for emotion classification was 92.36%. These results imply that for real-time, emotion-aware applications in real-world contexts, the suggested system is accurate and useful.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Autoencoder Approach to Speech Emotion Recognition and Intensity Estimation Using Latent Audio Features

  • Akshata Basapur,
  • Megha Bhajantri,
  • Sneha Tenginkai,
  • Pooja Karagi,
  • Sharada K. Shiragudikar

摘要

An essential first step in developing emotionally intelligent systems that can react to users more naturally is the recognition of emotions from speech. Human-computer interaction, healthcare, education, and customer service are just a few of the fields where this capability could enhance user experience. Pitch, tone, and rhythm are some of the components that speech uses to convey emotional information. These cues, when properly interpreted, enable machines to recognise human emotions and respond appropriately. In this paper, we present a deep learning-based method that estimates the strength of an emotion’s expression in speech in addition to identifying its type. The system uses Mel-Frequency Cepstral Coefficients (MFCCs) to first extract audio features in order to accomplish this. After that, an autoencoder compresses these features to create a lower-dimensional representation while preserving the emotional content. Two distinct modules receive this compressed data: one for categorising the emotion category and another for forecasting the emotion’s intensity on a continuous scale. In order to enhance performance and generalisation, the entire system is trained using a multi-task learning framework, which enables it to learn both tasks simultaneously. During training, class weighting and stratified sampling were used to address the dataset’s class imbalance. Furthermore, the suggested model outperformed baseline classifiers like SVM and CNN-LSTM in comparison. Strong results are obtained from our experiments on the RAVDESS dataset. The model’s mean absolute error (MAE) for intensity prediction was 0.1523 and its accuracy for emotion classification was 92.36%. These results imply that for real-time, emotion-aware applications in real-world contexts, the suggested system is accurate and useful.