Scholarlink

<p>The importance of understanding student engagement in learning with AI technologies is particularly relevant for the case of adaptive learning. Most current vision-based methods rely solely on appearance-based emotion detection using facial expressions. They do not consider the geometric and behavioral temporal dynamics that can occur in unconstrained webcam streaming. The main contribution of this paper is the formulation of a hierarchical method based on Transfomer-Graphs to perform engagement-aware affective inference. The dual architecture proposed here consists of a BEiT-based vision transformer and a graph attention network (GAT). The vision transformer is in charge of capturing the global facial features. The GAT is in charge of modeling the relational dependencies of the 468 facial landmarks. The affectNet model is trained to learn robust facial emotions with high representation capabilities. The emotions predicted in the model do not directly supervise affective engagement. Instead, via a rule-based method of affective model fusion the predicted emotions are engaged with relevant affective states. When combined with gaze-based attention and eye aspect ratio measures that reflect alertness, these predictors infer frame-level engagement. Temporal smothing is applied to these predictors to provide a video-level trajectory. Experimental results show that the proposed Transformer-Graph fusion architecture achieves robust facial emotion recognition with 88% accuracy and interpretable decision-making behaviour in real-world webcam settings. The method is suitable for webcam video. Most of the current methods rely on behavioral, or geometric features, whereas this method paramaterizes the matrix as a behavioral features overlay. This method does need to clarified as describe for the engagement dataset.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A hierarchical transformer–graph framework for explainable student engagement estimation in E-learning videos

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年5月13日

查看全文
  • Mohammed Hussain Alharbi,
  • Asim Suleman A. Alwabel,
  • Mansor Alohali,
  • Aidd Mohammed Almalki,
  • Mohd Fairuz Iskandar Othman,
  • Mohammad Nazir Ahmad

摘要

The importance of understanding student engagement in learning with AI technologies is particularly relevant for the case of adaptive learning. Most current vision-based methods rely solely on appearance-based emotion detection using facial expressions. They do not consider the geometric and behavioral temporal dynamics that can occur in unconstrained webcam streaming. The main contribution of this paper is the formulation of a hierarchical method based on Transfomer-Graphs to perform engagement-aware affective inference. The dual architecture proposed here consists of a BEiT-based vision transformer and a graph attention network (GAT). The vision transformer is in charge of capturing the global facial features. The GAT is in charge of modeling the relational dependencies of the 468 facial landmarks. The affectNet model is trained to learn robust facial emotions with high representation capabilities. The emotions predicted in the model do not directly supervise affective engagement. Instead, via a rule-based method of affective model fusion the predicted emotions are engaged with relevant affective states. When combined with gaze-based attention and eye aspect ratio measures that reflect alertness, these predictors infer frame-level engagement. Temporal smothing is applied to these predictors to provide a video-level trajectory. Experimental results show that the proposed Transformer-Graph fusion architecture achieves robust facial emotion recognition with 88% accuracy and interpretable decision-making behaviour in real-world webcam settings. The method is suitable for webcam video. Most of the current methods rely on behavioral, or geometric features, whereas this method paramaterizes the matrix as a behavioral features overlay. This method does need to clarified as describe for the engagement dataset.