Affective computing as an exciting interdisciplinary domain is delving into psychology, neuroscience, and computer science aims to enhance the field of human-computer interaction. It is enabling machines to act more naturally in recognizing and responding to human emotions. This paper marks the significant foray into multimodal emotion recognition approach in affective computing. It underscores the inherent constraints of unimodal systems while accentuating the escalating significance and applicability of multimodal datasets. This systematic review delves into advancement of multimodal emotion recognition approach, by analysing 120 papers from prominent databases such as Scopus, IEEE, Elsevier and Google Scholar. This comprehensive review remarks the advancement in multimodal emotion recognition approach, including data fusion methods, offering profound insights into their practical challenges. In conclusion, this review highlights critical concerns such as dataset diversity, cultural biases, and the complexity of integrating heterogeneous data from various modalities. A systematic review of existing literature underscores the continuous evolution of deep learning in refining data level fusion strategies as well as feature level fusion strategies. This will enhance the integration of multimodal data for emotion recognition in emotionally aware devices. However, a significant limitation lies in the challenge of effectively aligning heterogeneous data sources. This will continue to hinder progress and underscores the need for more advanced adaptive fusion techniques and robust modeling approaches.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Systematic Review of Multimodal Emotion Recognition Approaches for Affective Computing: Advances and Challenges

  • Saleha Mariyam,
  • Halima Sadia

摘要

Affective computing as an exciting interdisciplinary domain is delving into psychology, neuroscience, and computer science aims to enhance the field of human-computer interaction. It is enabling machines to act more naturally in recognizing and responding to human emotions. This paper marks the significant foray into multimodal emotion recognition approach in affective computing. It underscores the inherent constraints of unimodal systems while accentuating the escalating significance and applicability of multimodal datasets. This systematic review delves into advancement of multimodal emotion recognition approach, by analysing 120 papers from prominent databases such as Scopus, IEEE, Elsevier and Google Scholar. This comprehensive review remarks the advancement in multimodal emotion recognition approach, including data fusion methods, offering profound insights into their practical challenges. In conclusion, this review highlights critical concerns such as dataset diversity, cultural biases, and the complexity of integrating heterogeneous data from various modalities. A systematic review of existing literature underscores the continuous evolution of deep learning in refining data level fusion strategies as well as feature level fusion strategies. This will enhance the integration of multimodal data for emotion recognition in emotionally aware devices. However, a significant limitation lies in the challenge of effectively aligning heterogeneous data sources. This will continue to hinder progress and underscores the need for more advanced adaptive fusion techniques and robust modeling approaches.