<p>Generating high-quality 3D assets is a fundamental challenge in computer vision and graphics. While the field has progressed significantly from early VAE/GAN approaches through diffusion models and large reconstruction models, persistent limitations hinder widespread application. Specifically, achieving high geometric and appearance fidelity, intuitive user control, versatile multi-modal conditioning, and directly usable outputs (e.g., structured meshes) remains challenging for established paradigms. This paper surveys the evolution of deep generative models for 3D content creation, with a primary focus on emerging paradigms: autoregressive (AR) generation and Agent-driven approaches, poised to address aforementioned shortcomings. AR models generate assets sequentially (e.g., token-by-token or part-by-part), offering inherent potential for finer control, structured outputs, and integrating user guidance during the step-by-step process. Agent-driven methods, conversely, leverage the reasoning and linguistic capabilities of Large Language Models (LLMs), enabling intuitive and flexible 3D creation by decomposing complex tasks and utilizing external tools through multi-agent systems. We provide a comprehensive overview of these novel techniques, discuss their potential advantages over current methods, and outline key challenges and future directions towards more capable and intelligent 3D generation systems.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

3D asset generation: a survey of evolution towards autoregressive and agent-driven paradigms

  • Hongxing Fan,
  • Haohua Chen,
  • Zehuan Huang,
  • Ziwei Liu,
  • Lu Sheng

摘要

Generating high-quality 3D assets is a fundamental challenge in computer vision and graphics. While the field has progressed significantly from early VAE/GAN approaches through diffusion models and large reconstruction models, persistent limitations hinder widespread application. Specifically, achieving high geometric and appearance fidelity, intuitive user control, versatile multi-modal conditioning, and directly usable outputs (e.g., structured meshes) remains challenging for established paradigms. This paper surveys the evolution of deep generative models for 3D content creation, with a primary focus on emerging paradigms: autoregressive (AR) generation and Agent-driven approaches, poised to address aforementioned shortcomings. AR models generate assets sequentially (e.g., token-by-token or part-by-part), offering inherent potential for finer control, structured outputs, and integrating user guidance during the step-by-step process. Agent-driven methods, conversely, leverage the reasoning and linguistic capabilities of Large Language Models (LLMs), enabling intuitive and flexible 3D creation by decomposing complex tasks and utilizing external tools through multi-agent systems. We provide a comprehensive overview of these novel techniques, discuss their potential advantages over current methods, and outline key challenges and future directions towards more capable and intelligent 3D generation systems.