Text-Conditioned Zero-Shot 3D Avatar Creation and Animation
摘要
Creating 3D avatars has become essential in the digital landscape, yet the process remains extremely time-consuming and requires significant expertise. To make this technology accessible to a broader audience, this chapter discusses a novel framework called AvatarCLIP, which enables generation and animation of 3D avatars using text in a zero-shot manner. Unlike traditional software, which requires advanced technical knowledge, AvatarCLIP enables users with no expertise to design 3D avatars with specific shapes and textures and animate them using only descriptions in natural language. The core innovation lies in using the CLIP vision-language model to guide the generation of 3D human models, including their geometry, textures, and animations. Specifically, we begin by generating the 3D human shape through a VAE-based network, which is conditioned on textual prompts. Once the 3D shapes are produced, we apply a volume renderingVolume rendering approach to refine the geometry and textures. For animation, we introduce a novel method that combines motion priors learned in a motion VAE with CLIP-based guidance, enabling reference-driven motion synthesis. Comprehensive experiments demonstrate the robustness and versatility of AvatarCLIP, which can produce previously unseen avatars and animations with impressive zero-shot performance. Our code is publicly available at https://github.com/hongfz16/AvatarCLIP .