Recently, radar-based Hand Gesture Recognition (HGR) has gained increased attention in several applications involving contactless Human-Computer Interaction (HCI). Air-writing, as a definite HGR, requires real-time target positioning and trajectory tracking, followed by alphanumerical recognition via deep learning. Benefiting from the large virtual array provided by Multiple Input Multiple Output (MIMO) radar, this chapter presents an interferometry-based processing approach to acquire the subtle range and azimuth displacements of finger motions, thus fulfilling the tracking of the alphanumerical trajectory. A ResNet50 Convolutional Neural Network (CNN) trained with the trajectory is used to recognise the correct writings. Additionally, spatial interferometry is exploited to identify multiple strokes when writing specific characters, punctuations or words requiring non-contiguous hand motion articulation. This is achieved by utilizing the subtle elevation change induced by hand lifts among different strokes. Experimental results show that the proposed air-writing system performs rather well in sensing and tracking hand movements, achieving high recognition accuracy for different types of air writings.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Hand Air-Writing System Using MIMO Radar and Deep Learning

  • Xiangrong Wang,
  • Hengfeng Liu,
  • Moeness G. Amin,
  • Kaiquan Cai,
  • Yanbo Zhu

摘要

Recently, radar-based Hand Gesture Recognition (HGR) has gained increased attention in several applications involving contactless Human-Computer Interaction (HCI). Air-writing, as a definite HGR, requires real-time target positioning and trajectory tracking, followed by alphanumerical recognition via deep learning. Benefiting from the large virtual array provided by Multiple Input Multiple Output (MIMO) radar, this chapter presents an interferometry-based processing approach to acquire the subtle range and azimuth displacements of finger motions, thus fulfilling the tracking of the alphanumerical trajectory. A ResNet50 Convolutional Neural Network (CNN) trained with the trajectory is used to recognise the correct writings. Additionally, spatial interferometry is exploited to identify multiple strokes when writing specific characters, punctuations or words requiring non-contiguous hand motion articulation. This is achieved by utilizing the subtle elevation change induced by hand lifts among different strokes. Experimental results show that the proposed air-writing system performs rather well in sensing and tracking hand movements, achieving high recognition accuracy for different types of air writings.