The new system is an emotion recognition system that uses 3D images, audio, and video in a compound system to be more precise in the emotional tracking of people in culture-bound environments. Hybrid system integrates information from different modalities through the utilization of advanced deep learning techniques like CNNs, to perform the extraction of the spatial features and LSTMs, to model the temporal sequences. Preprocessing like reducing the audio noise and aligning the 3D facial landmarks followed by the frame selection as a data preprocessing technique. The model having deep layers applying the attention mechanisms to the process that is coved from data sources. For instance, datasets representing various emotional states in different cultures are used for the quoted purposes. The discussed reports indicate that there have been considerable improvements in recognition performance and generalization to different datasets, and as a result the system tends to become more and more applicable in areas such as human-computer interaction, mental disease diagnoses, the development of emotion-aware AI systems, as well as research on cross-culture behaviours.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Comprehensive Multimodal Approach to Emotion Detection Using Images, Audio and Video

  • P. V. Narasimha Raju,
  • S. Tarun,
  • V. Sai Pavan,
  • V. Murahari Krishna,
  • U. Abhidhar Kumar

摘要

The new system is an emotion recognition system that uses 3D images, audio, and video in a compound system to be more precise in the emotional tracking of people in culture-bound environments. Hybrid system integrates information from different modalities through the utilization of advanced deep learning techniques like CNNs, to perform the extraction of the spatial features and LSTMs, to model the temporal sequences. Preprocessing like reducing the audio noise and aligning the 3D facial landmarks followed by the frame selection as a data preprocessing technique. The model having deep layers applying the attention mechanisms to the process that is coved from data sources. For instance, datasets representing various emotional states in different cultures are used for the quoted purposes. The discussed reports indicate that there have been considerable improvements in recognition performance and generalization to different datasets, and as a result the system tends to become more and more applicable in areas such as human-computer interaction, mental disease diagnoses, the development of emotion-aware AI systems, as well as research on cross-culture behaviours.