Blink detection is crucial for applications such as cognitive load assessment, driver fatigue monitoring, and human-computer interaction. Traditional methods struggle with real-world challenges, such as head pose variations, occlusions, and changing lighting conditions. In this paper, we propose the OptFlowBlinkFormer model, which combines optical flow with static RGB features and leverages Transformer architecture to significantly improve blink detection performance in complex environments. The model introduces the Optical Flow Fusion Stem, which effectively combines spatiotemporal features, significantly enhancing performance in complex scenarios. We also introduce the EAMS dataset, a new benchmark designed to complement the HUST-LEBW dataset, containing diverse real-world movie scenes. Experimental results show that OptFlowBlinkFormer outperforms existing methods, achieving state-of-the-art F1 scores of 90.66% on HUST-LEBW and 94.87% on EAMS. This work advances blink detection by offering robust performance in real-world applications, with future work aimed at optimizing the model for more challenging environments and expanding datasets to improve generalization.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

OptFlowBlinkFormer: An Optical Flow Fusion Stem-Enhanced Transformer for Robust In-the-Wild Blink Detection

  • Wentao Qiu,
  • Xiaogang Xu

摘要

Blink detection is crucial for applications such as cognitive load assessment, driver fatigue monitoring, and human-computer interaction. Traditional methods struggle with real-world challenges, such as head pose variations, occlusions, and changing lighting conditions. In this paper, we propose the OptFlowBlinkFormer model, which combines optical flow with static RGB features and leverages Transformer architecture to significantly improve blink detection performance in complex environments. The model introduces the Optical Flow Fusion Stem, which effectively combines spatiotemporal features, significantly enhancing performance in complex scenarios. We also introduce the EAMS dataset, a new benchmark designed to complement the HUST-LEBW dataset, containing diverse real-world movie scenes. Experimental results show that OptFlowBlinkFormer outperforms existing methods, achieving state-of-the-art F1 scores of 90.66% on HUST-LEBW and 94.87% on EAMS. This work advances blink detection by offering robust performance in real-world applications, with future work aimed at optimizing the model for more challenging environments and expanding datasets to improve generalization.