Understanding human behavior is pivotal to the development of embodied artificial intelligence. Hand trajectories, as a critical medium of human interaction, offer a valuable lens through which to explore and interpret human actions. In this work, we propose a self-guided diffusion model for hand trajectory prediction (HTDiff). HTDiff consists of two main stages: The unconditional trajectory reconstruction stage learns from existing hand motion data to generate trajectory samples that conform to human motion patterns. In the conditional trajectory prediction stage, historical trajectories serve as contextual information to adaptively guide the pretrained reconstruction model in predicting future hand trajectories. Furthermore, we employ a Transformer architecture as the decoder within the diffusion model to fully exploit its strengths in sequence modeling, enabling the capture of temporal dependencies and complex motion patterns. Experiments conducted on public datasets reveal that HTDiff surpasses existing baseline methods in hand trajectory prediction, achieving the best performance across four evaluation metrics.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

HTDiff: Self-Guiding Diffusion Models for Hand Trajectory Prediction

  • Yu Liu,
  • Zipei Fan,
  • Tianlv Huang,
  • Wei Han,
  • Meiqi Zhou

摘要

Understanding human behavior is pivotal to the development of embodied artificial intelligence. Hand trajectories, as a critical medium of human interaction, offer a valuable lens through which to explore and interpret human actions. In this work, we propose a self-guided diffusion model for hand trajectory prediction (HTDiff). HTDiff consists of two main stages: The unconditional trajectory reconstruction stage learns from existing hand motion data to generate trajectory samples that conform to human motion patterns. In the conditional trajectory prediction stage, historical trajectories serve as contextual information to adaptively guide the pretrained reconstruction model in predicting future hand trajectories. Furthermore, we employ a Transformer architecture as the decoder within the diffusion model to fully exploit its strengths in sequence modeling, enabling the capture of temporal dependencies and complex motion patterns. Experiments conducted on public datasets reveal that HTDiff surpasses existing baseline methods in hand trajectory prediction, achieving the best performance across four evaluation metrics.