Human motion prediction is of great importance for artificial intelligence systems, particularly in fields like autonomous driving and human-computer interaction. Existing methods have achieved good results in simple motions with small changes but still struggle with coherence in predicting complex motions with large changes in a short time. Regarding the issue in current methods, this paper proposes a novel network composed of three modules: spatial encoders that captures correlations between joints, temporal encoders that models the movement patterns of joints across frames, and a multi-frame temporal encoder that learns high-level temporal correlations to enhance prediction coherence. In addition, we propose to use Weighted Mean Per-Joint Position Error (WMPJPE) as the loss function by considering that different joints of the human skeleton have different effects on the entire action. We evaluate our method on the Human3.6M, AMASS, and 3DPW datasets. The quantitative results indicate that our method notably outperforms other methods in the prediction range of 160ms to 560ms. The qualitative results show that our method can improve the coherence of prediction.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Spatio-Temporal Transformer for Enhancing the Coherence on 3D Human Motion Prediction

  • Miner Xie,
  • Lei Wang,
  • Feng Liu

摘要

Human motion prediction is of great importance for artificial intelligence systems, particularly in fields like autonomous driving and human-computer interaction. Existing methods have achieved good results in simple motions with small changes but still struggle with coherence in predicting complex motions with large changes in a short time. Regarding the issue in current methods, this paper proposes a novel network composed of three modules: spatial encoders that captures correlations between joints, temporal encoders that models the movement patterns of joints across frames, and a multi-frame temporal encoder that learns high-level temporal correlations to enhance prediction coherence. In addition, we propose to use Weighted Mean Per-Joint Position Error (WMPJPE) as the loss function by considering that different joints of the human skeleton have different effects on the entire action. We evaluate our method on the Human3.6M, AMASS, and 3DPW datasets. The quantitative results indicate that our method notably outperforms other methods in the prediction range of 160ms to 560ms. The qualitative results show that our method can improve the coherence of prediction.