The task of 3D hand gesture reconstruction from a single viewpoint is inherently challenging due to the frequent occurrence of occlusions that can obscure critical joint information. Traditional methods rely heavily on complete visual inputs, which are often unavailable in real-world scenarios. This paper introduces a novel transformer-based approach to 3D hand gesture reconstruction that effectively addresses the issue of occlusions. The proposed method leverages the transformer architecture to model both temporal and spatial dependencies and incorporates a masking operation to simulate occlusion scenarios. By simulating occlusions during training, the model learns to infer missing joint information effectively. Additionally, the model integrates anatomical prior knowledge of hand gestures to guide the prediction of occluded joint positions, enhancing the model’s performance in complex environments.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

3D Hand Gesture Reconstruction in Occluded Environments Using a Transformer-Based Approach

  • Changliang Zheng,
  • Hui Du,
  • Kai Zhao,
  • Taizhong Zhang,
  • Junna Gao

摘要

The task of 3D hand gesture reconstruction from a single viewpoint is inherently challenging due to the frequent occurrence of occlusions that can obscure critical joint information. Traditional methods rely heavily on complete visual inputs, which are often unavailable in real-world scenarios. This paper introduces a novel transformer-based approach to 3D hand gesture reconstruction that effectively addresses the issue of occlusions. The proposed method leverages the transformer architecture to model both temporal and spatial dependencies and incorporates a masking operation to simulate occlusion scenarios. By simulating occlusions during training, the model learns to infer missing joint information effectively. Additionally, the model integrates anatomical prior knowledge of hand gestures to guide the prediction of occluded joint positions, enhancing the model’s performance in complex environments.