A comprehensive soft spatial control of human motion generation via dual stage diffusion
摘要
The text-driven motion generation (T2M) task aims to utilize natural language to regulate the generation of realistic human motion through diverse computer vision techniques. Existing works primarily explore unilateral or bilateral regulation, as well as rigid and non-user-friendly hard controls, leading to incomprehensive and restricted control over motion generation. Our work, however, comprehensively considers control of three elements: trajectory, rotation-invariant pose, and global rotation in a soft control manner. To realize this, we use a dual-branch diffusion-based module that can decouple and coordinate these three key control factors through flexible soft controls. Specifically, these three independent soft control signal streams are injected into the dual-branch neural network as the control signal of diffusion process, which comprehensively guides the motion generation. These three soft control signals further exert influence in the form of soft spatial regression guidance. Additionally, we adopt a coarse-to-fine paradigm that first generating preliminary motions based on soft controls, which can serve as informative prior knowledge and additional regression guidance to further enhance performance. By providing “weak yet direct” control, the proposed framework facilitates flexible guidance in motion generation while maintaining the vividness and naturalness of the generated motion. Experimental results demonstrate that our framework performs well and outperforms other baseline methods.