Facial Emotion Recognition (FER) and engagement analysis are critical for applications in human–computer interaction and health care. This study presents a robust model leveraging Vision Transformers (ViTs) for emotion classification and K-Means clustering for engagement level categorization. ViTs process images as sequences of patches, capturing both global and local dependencies for superior feature extraction, while K-Means clustering classifies engagement levels into three categories: highly engaged, moderately engaged, and barely engaged. The unsupervised nature of K-Means enables dynamic adaptation to varying user interactions without requiring manually labeled data. Experimental results on the AffectNet dataset demonstrate an 84% accuracy in emotion recognition, with engagement classification validated by silhouette scores exceeding 0.78. This approach introduces a novel methodology to classify emotional engagement, advancing FER systems and providing valuable insights into real-time human behavior analysis.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Facial Emotion Recognition and Engagement Analysis Using Vision Transformers

  • K. Koustubh,
  • Akshay Pujari,
  • N. Amoghmanoranjith,
  • P. L. Sujan,
  • Channabasappa Muttal,
  • Sneha Varur

摘要

Facial Emotion Recognition (FER) and engagement analysis are critical for applications in human–computer interaction and health care. This study presents a robust model leveraging Vision Transformers (ViTs) for emotion classification and K-Means clustering for engagement level categorization. ViTs process images as sequences of patches, capturing both global and local dependencies for superior feature extraction, while K-Means clustering classifies engagement levels into three categories: highly engaged, moderately engaged, and barely engaged. The unsupervised nature of K-Means enables dynamic adaptation to varying user interactions without requiring manually labeled data. Experimental results on the AffectNet dataset demonstrate an 84% accuracy in emotion recognition, with engagement classification validated by silhouette scores exceeding 0.78. This approach introduces a novel methodology to classify emotional engagement, advancing FER systems and providing valuable insights into real-time human behavior analysis.