3D asset generation: a survey of evolution towards autoregressive and agent-driven paradigms
摘要
Generating high-quality 3D assets is a fundamental challenge in computer vision and graphics. While the field has progressed significantly from early VAE/GAN approaches through diffusion models and large reconstruction models, persistent limitations hinder widespread application. Specifically, achieving high geometric and appearance fidelity, intuitive user control, versatile multi-modal conditioning, and directly usable outputs (e.g., structured meshes) remains challenging for established paradigms. This paper surveys the evolution of deep generative models for 3D content creation, with a primary focus on emerging paradigms: autoregressive (AR) generation and Agent-driven approaches, poised to address aforementioned shortcomings. AR models generate assets sequentially (e.g., token-by-token or part-by-part), offering inherent potential for finer control, structured outputs, and integrating user guidance during the step-by-step process. Agent-driven methods, conversely, leverage the reasoning and linguistic capabilities of Large Language Models (LLMs), enabling intuitive and flexible 3D creation by decomposing complex tasks and utilizing external tools through multi-agent systems. We provide a comprehensive overview of these novel techniques, discuss their potential advantages over current methods, and outline key challenges and future directions towards more capable and intelligent 3D generation systems.