Gestures exhibit sparse joint variations and different time scales, making local dynamic analysis and global spatio-temporal modeling important. Path signature provides mathematical and dynamic analysis of joint trajectories to assist in spatio-temporal modeling. However, previous methods relied on predefined local spatio-temporal joint trajectories, also known as paths. This limitation makes it challenging to directly capture the dynamics of the entire gesture and adapt to varying scales of gesture changes. In this work, we construct the Adaptive Global Gesture Path and extract its signature features as gesture representations. Specifically, we designed global branch to model the global spatio-temporal variation relationship of joints. The dynamic branch is based on the proposed Motion Guided Cluster Attention Block, which emphasizes joints exhibiting similar motion patterns. Combining two branches, the predicted dynamic and global score can distinguish key joints at different times to construct the Adaptive Global Gesture Path that condensely represents the entire gesture. We conducted experiments on the ChaLearn2013 and WLASL datasets, and achieved the state-of-the-art results with much smaller model size.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Adaptive Global Gesture Paths and Signature Features for Skeleton-based Gesture Recognition

  • Dongzi Shi,
  • Xin Zhang,
  • Jiale Cheng,
  • Tong Xiong,
  • Hao Ni

摘要

Gestures exhibit sparse joint variations and different time scales, making local dynamic analysis and global spatio-temporal modeling important. Path signature provides mathematical and dynamic analysis of joint trajectories to assist in spatio-temporal modeling. However, previous methods relied on predefined local spatio-temporal joint trajectories, also known as paths. This limitation makes it challenging to directly capture the dynamics of the entire gesture and adapt to varying scales of gesture changes. In this work, we construct the Adaptive Global Gesture Path and extract its signature features as gesture representations. Specifically, we designed global branch to model the global spatio-temporal variation relationship of joints. The dynamic branch is based on the proposed Motion Guided Cluster Attention Block, which emphasizes joints exhibiting similar motion patterns. Combining two branches, the predicted dynamic and global score can distinguish key joints at different times to construct the Adaptive Global Gesture Path that condensely represents the entire gesture. We conducted experiments on the ChaLearn2013 and WLASL datasets, and achieved the state-of-the-art results with much smaller model size.