Scholarlink

<p>The task of student engagement recognition aims to classify given facial videos into several basic levels of engagement. The key challenge lies in how to effectively integrate facial features from each frame to improve classification performance. To address this issue, we propose a novel frame attention neural network (FANN), which aims to automatically highlight discriminative video frames to enhance the performance of student engagement recognition. FANN mainly consists of two parts: a data processing module (DPM) and a frame attention module (FAM). Specifically, DPM employs balanced binary tree sampling technology to generate new data samples, thereby increasing the quantity of the dataset. Then, FAM captures potential intra-frame and inter-frame attention weights to adaptively aggregate video features. To achieve this, a self-attention kernel (SA) and a relation-attention kernel (RA) are developed within FAM. In addition, we propose an equalization loss to mitigate the class imbalance issue in the dataset. The experimental results on the DAiSEE dataset and the EmotiW-EP dataset demonstrate that the proposed FANN method achieves superior performance compared to other deep learning approaches. This indicates that FANN can adaptively aggregate feature vectors to form more discriminative video representations, thereby improving the performance of student engagement recognition. Our codes will be available at <a href="https://github.com/Autonomy-Life/FANN.git">https://github.com/Autonomy-Life/FANN.git</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

FANN: a novel frame attention neural network for student engagement recognition in facial video

  • 期刊论文

  • 发表日期：

    2025年1月27日

查看全文
  • Hu Wang,
  • Hong-Mei Sun,
  • Wen-Long Zhang,
  • Yu-Xiang Chen,
  • Rui-Sheng Jia

摘要

The task of student engagement recognition aims to classify given facial videos into several basic levels of engagement. The key challenge lies in how to effectively integrate facial features from each frame to improve classification performance. To address this issue, we propose a novel frame attention neural network (FANN), which aims to automatically highlight discriminative video frames to enhance the performance of student engagement recognition. FANN mainly consists of two parts: a data processing module (DPM) and a frame attention module (FAM). Specifically, DPM employs balanced binary tree sampling technology to generate new data samples, thereby increasing the quantity of the dataset. Then, FAM captures potential intra-frame and inter-frame attention weights to adaptively aggregate video features. To achieve this, a self-attention kernel (SA) and a relation-attention kernel (RA) are developed within FAM. In addition, we propose an equalization loss to mitigate the class imbalance issue in the dataset. The experimental results on the DAiSEE dataset and the EmotiW-EP dataset demonstrate that the proposed FANN method achieves superior performance compared to other deep learning approaches. This indicates that FANN can adaptively aggregate feature vectors to form more discriminative video representations, thereby improving the performance of student engagement recognition. Our codes will be available at https://github.com/Autonomy-Life/FANN.git.