<p>Recent advances in image/video generative models and learnable dynamic scene representations have enabled the easy creation of 4D scenes. However, the question of how to achieve more controlled motion generation rather than providing high-level text prompts is not well explored. This paper presents an innovative method that leverages user-specified action paths to guide the 4D scene generation. The challenges of motion-controlled 4D scene generation mainly lie in the temporal variations of the motion and the high requirement for temporal consistency. To address these, our method incorporates an adaptive motion alignment module to enhance the naturalness of the generated motion, which dynamically synchronizes motions in the action path domain with their corresponding contents in the time domain. Additionally, to mitigate artifacts caused by excessive motion, we introduce a progressive motion drive module to ensure high-quality outputs by enhancing the control of motion. Experimental results demonstrate the superiority of our method in performing motion-controlled 4D scene generation.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Motion-driven 4D scene generation

  • Guo-Wei Yang,
  • Qun-Ce Xu,
  • Zhao Wei,
  • Shi-Min Hu

摘要

Recent advances in image/video generative models and learnable dynamic scene representations have enabled the easy creation of 4D scenes. However, the question of how to achieve more controlled motion generation rather than providing high-level text prompts is not well explored. This paper presents an innovative method that leverages user-specified action paths to guide the 4D scene generation. The challenges of motion-controlled 4D scene generation mainly lie in the temporal variations of the motion and the high requirement for temporal consistency. To address these, our method incorporates an adaptive motion alignment module to enhance the naturalness of the generated motion, which dynamically synchronizes motions in the action path domain with their corresponding contents in the time domain. Additionally, to mitigate artifacts caused by excessive motion, we introduce a progressive motion drive module to ensure high-quality outputs by enhancing the control of motion. Experimental results demonstrate the superiority of our method in performing motion-controlled 4D scene generation.