This study performs emotion classification in older adults through the analysis of multimodal conversations that integrate data from three modalities: text, audio, and video. This approach is essential in contexts where communication plays a central role, as it enables a more accurate and empathetic interpretation of emotions. During the development process, Natural Language Processing, Computer Vision, and Audio Signal Processing techniques were employed to extract relevant features from each modality. Based on these features, three classification models were built and evaluated: a Multilayer Perceptron (MLP), a Support Vector Machine (SVM), and Random Forests (RF). The models were assessed using the F1-score metric, achieving competitive results exceeding 90%. Beyond the numerical performance, this work highlights the importance of multimodal integration for understanding the complexities of human interaction in aging populations. The analysis aims to contribute to the development of more effective systems for emotion detection, with potential applications in healthcare, social support, and human-computer interaction.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Analysis of Conversations in Adults Using Multimodal Data Processing

  • Carolina Naomi Alvarez-Peralta,
  • Janet Badillo-Tecuapacho,
  • Eric Ramos-Aguilar,
  • Daniel Sánchez-Ruiz,
  • Ricardo Ramos-Aguilar

摘要

This study performs emotion classification in older adults through the analysis of multimodal conversations that integrate data from three modalities: text, audio, and video. This approach is essential in contexts where communication plays a central role, as it enables a more accurate and empathetic interpretation of emotions. During the development process, Natural Language Processing, Computer Vision, and Audio Signal Processing techniques were employed to extract relevant features from each modality. Based on these features, three classification models were built and evaluated: a Multilayer Perceptron (MLP), a Support Vector Machine (SVM), and Random Forests (RF). The models were assessed using the F1-score metric, achieving competitive results exceeding 90%. Beyond the numerical performance, this work highlights the importance of multimodal integration for understanding the complexities of human interaction in aging populations. The analysis aims to contribute to the development of more effective systems for emotion detection, with potential applications in healthcare, social support, and human-computer interaction.