Generating physically plausible and controllable 3D dynamic scenes remains a central challenge in computer graphics and vision. While existing methods show promise, they often struggle to reconcile the demands of realistic physics simulation with the need for intuitive, user-driven control. We present a novel framework that synergizes generative AI with robust physics-based simulation, all grounded in a highly efficient 3D Gaussian representation. Our method employs a multimodal pipeline that integrates state-of-the-art models for both foreground asset creation and background scene understanding. A core innovation is a Large Language Model (LLM) agent that functions as a semantic actuator, translating natural language commands into physical forces to enable intuitive, real-time control over scene dynamics. To ensure stability and physical realism, we introduce a novel simulation engine featuring spatiotemporally consistent grid decoupling to preserve static background integrity and a high-fidelity collision model based on Signed Distance Fields (SDFs). Comprehensive evaluations on benchmarks like ScanNet demonstrate state-of-the-art performance, achieving a rendering quality of 28.7 PSNR while operating at 17.3 FPS. Experiments confirm that our framework provides a practical and powerful solution for interactive dynamic scene generation.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

GauScene: Physically Plausible Scene Generation via Language-Guided 3D Gaussian Interaction

  • Fang Liang,
  • Jianshu Guo,
  • Xuexiang Wen,
  • Wenhao Hu,
  • Xiang Ye,
  • Gaoang Wang

摘要

Generating physically plausible and controllable 3D dynamic scenes remains a central challenge in computer graphics and vision. While existing methods show promise, they often struggle to reconcile the demands of realistic physics simulation with the need for intuitive, user-driven control. We present a novel framework that synergizes generative AI with robust physics-based simulation, all grounded in a highly efficient 3D Gaussian representation. Our method employs a multimodal pipeline that integrates state-of-the-art models for both foreground asset creation and background scene understanding. A core innovation is a Large Language Model (LLM) agent that functions as a semantic actuator, translating natural language commands into physical forces to enable intuitive, real-time control over scene dynamics. To ensure stability and physical realism, we introduce a novel simulation engine featuring spatiotemporally consistent grid decoupling to preserve static background integrity and a high-fidelity collision model based on Signed Distance Fields (SDFs). Comprehensive evaluations on benchmarks like ScanNet demonstrate state-of-the-art performance, achieving a rendering quality of 28.7 PSNR while operating at 17.3 FPS. Experiments confirm that our framework provides a practical and powerful solution for interactive dynamic scene generation.