Scholarlink

This article describes a method for the engagement prediction task, which is a sub-challenge of the 10th Emotion Recognition in the Wild Challenge (EmotiW) 2024. Our method includes three basic steps: feature extraction, model training, and model integration. In the first step, we extracted optical flow features from the video and used OpenFace to extract eye gaze data. In the second step, we utilized a Dual-Stream Spatiotemporal Transformer (DST) and a Bi-LSTM to capture temporal information and learn from the features obtained in the previous step. In the final step, we fused the results from the Dual-Stream Spatiotemporal Transformer and the Bi-LSTM to achieve better performance. Ultimately, our method achieved an accuracy of 66.29% on the validation set.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Multimodal Engagement Recognition by Fusing Transformer and Bi-LSTM

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Yan Zhao,
  • Jinke Xu,
  • Xiaohua Huang

摘要

This article describes a method for the engagement prediction task, which is a sub-challenge of the 10th Emotion Recognition in the Wild Challenge (EmotiW) 2024. Our method includes three basic steps: feature extraction, model training, and model integration. In the first step, we extracted optical flow features from the video and used OpenFace to extract eye gaze data. In the second step, we utilized a Dual-Stream Spatiotemporal Transformer (DST) and a Bi-LSTM to capture temporal information and learn from the features obtained in the previous step. In the final step, we fused the results from the Dual-Stream Spatiotemporal Transformer and the Bi-LSTM to achieve better performance. Ultimately, our method achieved an accuracy of 66.29% on the validation set.