<p>Emotion recognition is pivotal in advancing human-computer interaction, with transformative applications in education, healthcare, and social analysis. Traditional models predominantly rely on facial expressions, often neglecting the critical roles of contextual and bodily cues. In this study, we propose mCFA, a novel context-aware emotion recognition model that integrates facial, body, and contextual cues through a two-level attention mechanism. First, cross-attention modules dynamically learn inter-modality interactions; second, adaptive fusion assigns weights to each modality based on its contribution. Evaluated on the EMOTIC dataset, mCFA achieves a mean Average Precision (mAP) of 28.77%, and an Accuracy of 85.53% on the CAER-S dataset, outperforming several state-of-the-art models. Our approach demonstrates robust performance, particularly in the presence of occlusion or missing visual cues, underscoring the effectiveness of attention-driven fusion for emotion understanding.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Attention mechanisms for context-aware emotion recognition

  • Hung Nguyen,
  • Nha Tran,
  • Minh Nguyen,
  • Phi Ta,
  • Hien D. Nguyen

摘要

Emotion recognition is pivotal in advancing human-computer interaction, with transformative applications in education, healthcare, and social analysis. Traditional models predominantly rely on facial expressions, often neglecting the critical roles of contextual and bodily cues. In this study, we propose mCFA, a novel context-aware emotion recognition model that integrates facial, body, and contextual cues through a two-level attention mechanism. First, cross-attention modules dynamically learn inter-modality interactions; second, adaptive fusion assigns weights to each modality based on its contribution. Evaluated on the EMOTIC dataset, mCFA achieves a mean Average Precision (mAP) of 28.77%, and an Accuracy of 85.53% on the CAER-S dataset, outperforming several state-of-the-art models. Our approach demonstrates robust performance, particularly in the presence of occlusion or missing visual cues, underscoring the effectiveness of attention-driven fusion for emotion understanding.