In current education scenarios, the monitoring of attention levels among students constitutes an important aspect in increasing teaching effectiveness. In this paper, we bring forth a system for recognizing classroom behavior based on visible cues that employs both gaze direction detection and posture estimation. System architecture constitutes a combination of three primary modules, namely Faster R-CNN for student recognition, HRNet for extracting posture, and L2CS-Net for estimating gaze. Features obtained from the HRNet are embedded in a convolutional neural network (CNN) that places student activity in four respective classes: looking, asking, boring, and bowing. Experimental evaluations utilizing a real-world testing set of 832 images establish the system as having remarkable classification capacity, obtaining an average F1-score of 0.9925. Notably, inquiring and boring obtained perfect precision and recall values (1.0), which establish superior recognition mastery. Bending behavior achieved a recall of 0.98 and an F1-score of 0.99, and the stare feature set obtained equivalent values, including a precision value of 0.98 and a recall value of 0.99. Utilization of pre-trained deep learning models efficiently mitigates the workload of data annotation costs while, in addition, improving functional practicability. Incorporating this approach constitutes a prospective measure toward implementation in smart classroom technologies, allowing teachers to perceive student behavior in a salient, accurate, and real-time format.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Real-Time Student Behavior Recognition in Classroom Using Pose Estimation and Gaze Analysis

  • Huu-Huy Ngo,
  • Hung Linh Le,
  • Nguyen Duy Minh,
  • Duc-Tuong Duong,
  • Tien-Khai Vu

摘要

In current education scenarios, the monitoring of attention levels among students constitutes an important aspect in increasing teaching effectiveness. In this paper, we bring forth a system for recognizing classroom behavior based on visible cues that employs both gaze direction detection and posture estimation. System architecture constitutes a combination of three primary modules, namely Faster R-CNN for student recognition, HRNet for extracting posture, and L2CS-Net for estimating gaze. Features obtained from the HRNet are embedded in a convolutional neural network (CNN) that places student activity in four respective classes: looking, asking, boring, and bowing. Experimental evaluations utilizing a real-world testing set of 832 images establish the system as having remarkable classification capacity, obtaining an average F1-score of 0.9925. Notably, inquiring and boring obtained perfect precision and recall values (1.0), which establish superior recognition mastery. Bending behavior achieved a recall of 0.98 and an F1-score of 0.99, and the stare feature set obtained equivalent values, including a precision value of 0.98 and a recall value of 0.99. Utilization of pre-trained deep learning models efficiently mitigates the workload of data annotation costs while, in addition, improving functional practicability. Incorporating this approach constitutes a prospective measure toward implementation in smart classroom technologies, allowing teachers to perceive student behavior in a salient, accurate, and real-time format.