Monitoring student facial expressions for interest is crucial for modern learning, offering insights into their cognitive and emotional states so instructors can adapt teaching and provide support. Traditional methods are slow and subjective. The existing deep learning methods often relies on internet-based datasets and simplistic emotion-to-engagement mappings. They fail to capture temporal dynamics, and limits their real-world classroom applicability. This study presents a framework integrating a Video Swin Transformer to extract spatio-temporal facial features and classify seven basic emotions. Subsequently, a frequency-weighted mapping function is applied to convert the resulting emotion sequences into four discrete levels of student engagement. Besides, this study also introduces an open dataset, HCMUE-SEED, comprising video recordings of 35 students participating in authentic in-class learning activities. The proposed emotion recognition model achieves 83.41% accuracy, and the end-to-end engagement estimation reaches 78.83% accuracy. It is effective and potential to inform real-time, objective classroom analytics for instructors.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Student Engagement Estimation in Classroom Videos Using Video Swin Transformer

  • Nha Tran,
  • Hung Nguyen,
  • Dat Ly,
  • Hung Q. Nguyen,
  • Anh Tran,
  • Hien D. Nguyen

摘要

Monitoring student facial expressions for interest is crucial for modern learning, offering insights into their cognitive and emotional states so instructors can adapt teaching and provide support. Traditional methods are slow and subjective. The existing deep learning methods often relies on internet-based datasets and simplistic emotion-to-engagement mappings. They fail to capture temporal dynamics, and limits their real-world classroom applicability. This study presents a framework integrating a Video Swin Transformer to extract spatio-temporal facial features and classify seven basic emotions. Subsequently, a frequency-weighted mapping function is applied to convert the resulting emotion sequences into four discrete levels of student engagement. Besides, this study also introduces an open dataset, HCMUE-SEED, comprising video recordings of 35 students participating in authentic in-class learning activities. The proposed emotion recognition model achieves 83.41% accuracy, and the end-to-end engagement estimation reaches 78.83% accuracy. It is effective and potential to inform real-time, objective classroom analytics for instructors.