Scholarlink

<p>Real-time assessment and enhancement of student engagement remains a significant challenge in higher education. Existing AI-based systems predominantly offer descriptive analytics without closed-loop feedback and rely on limited data modalities. We propose a novel AI-based multimodal framework that integrates visual, audio, physiological, and behavioral data streams to detect student engagement in real-time using a temporal Bidirectional LSTM (BiLSTM) model. The system features a closed-loop feedback mechanism that triggers personalised interventions when engagement drops below adaptive thresholds. The framework is rigorously evaluated on four public benchmark datasets (DAiSEE, CMOSE, RoomReader, DIPSER) covering diverse learning environments. Our approach achieves state-of-the-art performance with a macro-F1 score of up to 0.78 and 82% accuracy on the CMOSE dataset, outperforming vision-only models by +0.10 macro-F1, late-fusion baselines by +0.07 macro-F1, and log-only models by up to +0.22 macro-F1. Ablation studies reveal physiological signals as the most significant modality (35% contribution), followed by visual (28%) and audio (22%) cues. The closed-loop system successfully triggers context-aware interventions during engagement drops in simulated sessions. This work bridges the gap between descriptive analytics and actionable pedagogy, offering a scalable, ethically-aware framework for proactive learning support in university settings.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A multimodal AI framework for real-time student engagement detection and adaptive feedback in higher education

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年3月18日

查看全文
  • Zhao Dan,
  • Zhao Yali,
  • Zhao Tong

摘要

Real-time assessment and enhancement of student engagement remains a significant challenge in higher education. Existing AI-based systems predominantly offer descriptive analytics without closed-loop feedback and rely on limited data modalities. We propose a novel AI-based multimodal framework that integrates visual, audio, physiological, and behavioral data streams to detect student engagement in real-time using a temporal Bidirectional LSTM (BiLSTM) model. The system features a closed-loop feedback mechanism that triggers personalised interventions when engagement drops below adaptive thresholds. The framework is rigorously evaluated on four public benchmark datasets (DAiSEE, CMOSE, RoomReader, DIPSER) covering diverse learning environments. Our approach achieves state-of-the-art performance with a macro-F1 score of up to 0.78 and 82% accuracy on the CMOSE dataset, outperforming vision-only models by +0.10 macro-F1, late-fusion baselines by +0.07 macro-F1, and log-only models by up to +0.22 macro-F1. Ablation studies reveal physiological signals as the most significant modality (35% contribution), followed by visual (28%) and audio (22%) cues. The closed-loop system successfully triggers context-aware interventions during engagement drops in simulated sessions. This work bridges the gap between descriptive analytics and actionable pedagogy, offering a scalable, ethically-aware framework for proactive learning support in university settings.