PaMG: adaptive part-based motion generation and editing from text
摘要
Synthesis of 3D human motion based on text descriptions is a task with broad applications but significant challenges due to the data distribution differences between these two modalities. Unlike previous methods that directly model full-body movements, we introduce PaMG, a framework that leverages the local features of body part motions, achieving better performance in both global and local text-driven human motion generation and editing. In this paper, we first decouple the body joints into multiple parts and use hierarchical quantization to extract their local features, capturing motion details of different body movements, which are then stored as a series of discrete motion tokens. These part tokens are then decoded to generate input 3D human motion to validate their effectiveness. To address the motion consistency issue across different body parts, we introduce Part-motion Generative Pre-trained Transformer (GPT) to explore the latent connection between different body part motions, which integrates text features with information from different body parts to establish the mapping relationship between language and movements. Furthermore, PaMG can not only generate global human body movements based on text but also support local part motion editing, which is not available in other methods. Extensive experiments demonstrate that our method outperforms existing methods, validating the effectiveness of our approach.