Scholarlink

<p>Diffusion model has demonstrated remarkable capability in video generation, which further sparks interest in introducing trajectory control into the generation process. While existing works mainly focus on training-based methods (<i>e.g.</i>, conditional adapter), we argue that diffusion model itself allows decent control over the generated content without requiring any training. In this study, we introduce a tuning-free framework to achieve trajectory-controllable video generation, by imposing guidance on both noise construction and attention computation. Specifically, <b>1)</b> we first show several instructive phenomena and analyze how initial noises influence the motion trajectory of generated content. <b>2)</b> Subsequently, we propose <b>FreeTraj</b>, a tuning-free approach that enables trajectory control by modifying noise sampling and attention mechanisms. <b>3)</b> Furthermore, we extend FreeTraj to facilitate longer and larger video generation with controllable trajectories. Equipped with these designs, users have the flexibility to provide trajectories manually or opt for trajectories automatically generated by the LLM trajectory planner. Extensive experiments validate the efficacy of our approach in enhancing the trajectory controllability of video diffusion models. Generated video samples are available at the website: <a href="http://haonanqiu.com/projects/FreeTraj.html">http://haonanqiu.com/projects/FreeTraj.html</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

FreeTraj: Tuning-Free Trajectory Control via Noise Guided Video Diffusion

  • 期刊论文

  • 发表日期：

    2026年3月7日

查看全文
  • Haonan Qiu,
  • Zhaoxi Chen,
  • Zhouxia Wang,
  • Yingqing He,
  • Menghan Xia,
  • Ziwei Liu

摘要

Diffusion model has demonstrated remarkable capability in video generation, which further sparks interest in introducing trajectory control into the generation process. While existing works mainly focus on training-based methods (e.g., conditional adapter), we argue that diffusion model itself allows decent control over the generated content without requiring any training. In this study, we introduce a tuning-free framework to achieve trajectory-controllable video generation, by imposing guidance on both noise construction and attention computation. Specifically, 1) we first show several instructive phenomena and analyze how initial noises influence the motion trajectory of generated content. 2) Subsequently, we propose FreeTraj, a tuning-free approach that enables trajectory control by modifying noise sampling and attention mechanisms. 3) Furthermore, we extend FreeTraj to facilitate longer and larger video generation with controllable trajectories. Equipped with these designs, users have the flexibility to provide trajectories manually or opt for trajectories automatically generated by the LLM trajectory planner. Extensive experiments validate the efficacy of our approach in enhancing the trajectory controllability of video diffusion models. Generated video samples are available at the website: http://haonanqiu.com/projects/FreeTraj.html.