This research aims to develop a gesture recognition system based on deep learning that is eligible for human-computer interaction in the industry. By integrating a feedforward hand landmarks model with CNN-LSTM architecture designed by ourselves, the system can capture feature of both static and dynamic gestures, then offer real-time and precise gesture recognition capabilities. We deployed the system on the NVIDIA Jetson Nano embedded system, to enhance efficiency and reduce response time by its GPU. An impressive accuracy of 99.04% is shown in the experiments, besides, the system performs exceptionally well under various situations, such as different backgrounds and lighting environments. The result proves the validity of our CNN-LSTM model and emphasizes its ability for identical use in real scenes. With the growth of reliable interactive interface demands, this research provides a solution to eliminating the limitations of traditional human-computer interaction way, furthermore, lights up the future for improving user experiences and productivity.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Spatiotemporal Gesture Recognition System Based on Landmarks and CNN-LSTM

  • Shao-Hsuan Lin,
  • Yu-Chiao Jhuang,
  • Chiao Zha,
  • Yen-Ting Chen,
  • Jeng-Shiou Leu

摘要

This research aims to develop a gesture recognition system based on deep learning that is eligible for human-computer interaction in the industry. By integrating a feedforward hand landmarks model with CNN-LSTM architecture designed by ourselves, the system can capture feature of both static and dynamic gestures, then offer real-time and precise gesture recognition capabilities. We deployed the system on the NVIDIA Jetson Nano embedded system, to enhance efficiency and reduce response time by its GPU. An impressive accuracy of 99.04% is shown in the experiments, besides, the system performs exceptionally well under various situations, such as different backgrounds and lighting environments. The result proves the validity of our CNN-LSTM model and emphasizes its ability for identical use in real scenes. With the growth of reliable interactive interface demands, this research provides a solution to eliminating the limitations of traditional human-computer interaction way, furthermore, lights up the future for improving user experiences and productivity.