3D human motion generation plays a crucial role in many applications. However, in previous studies, motion generation algorithms often focused on a limited number of motion categories or required complex methods to control the desired outputs. The goal of this chapter is to address the question of how to use natural language as the sole control signal to generate diverse and realistic motion sequences while enabling users to perform fine-grained control. This form of control greatly lowers the threshold for using this technology, allowing even layman users to benefit from the technology. Specifically, this chapter first uses MotionDiffuse as an example to introduce a classic text-driven motion generation pipeline design and studies how to fully integrate the text modality with the motion modality. Then, the chapter discusses how to leverage the concept of retrieval-augmented generation to further enhance the model’s generation performance. Finally, this chapter introduces two effective solutions to solve the problem of fine-grained motion generation and demonstrates how such generation capability can benefit real-world applications.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Text-Driven 3D Human Motion Generation

  • Mingyuan Zhang,
  • Zhongang Cai,
  • Liang Pan,
  • Chenyang Gu,
  • Fangzhou Hong,
  • Xinying Guo,
  • Jiawei Ren,
  • Lei Yang,
  • Ziwei Liu

摘要

3D human motion generation plays a crucial role in many applications. However, in previous studies, motion generation algorithms often focused on a limited number of motion categories or required complex methods to control the desired outputs. The goal of this chapter is to address the question of how to use natural language as the sole control signal to generate diverse and realistic motion sequences while enabling users to perform fine-grained control. This form of control greatly lowers the threshold for using this technology, allowing even layman users to benefit from the technology. Specifically, this chapter first uses MotionDiffuse as an example to introduce a classic text-driven motion generation pipeline design and studies how to fully integrate the text modality with the motion modality. Then, the chapter discusses how to leverage the concept of retrieval-augmented generation to further enhance the model’s generation performance. Finally, this chapter introduces two effective solutions to solve the problem of fine-grained motion generation and demonstrates how such generation capability can benefit real-world applications.