Scholarlink

<p>Controlling camera pose in video diffusion models is essential for novel-view video rendering, yet existing approaches struggle to achieve precise control. Methods that directly inject numerical camera parameters into the diffusion backbone often fail to capture subtle viewpoint variations and lead to structural distortions or visual artifacts. To overcome these limitations, we propose UniCam, a unified framework that introduces a temporally coherent stochastic representation, termed CameraNoise, warped from camera intrinsic and extrinsic parameters. Unlike conventional approaches, CameraNoise embeds camera poses directly into the noise space. This makes our approach independent of scene appearance while faithfully encoding camera motion. Specifically, we introduce a novel Geometry-guided Reprojection Flow along with a CameraNoise warping algorithm, which jointly preserves the Gaussian prior of diffusion and ensures consistent noise propagation under camera transformations. By integrating CameraNoise into the diffusion process, the UniCam framework produces stable, high-quality videos with precise camera control across text-to-video, image-to-video, and video-to-video generation tasks. Extensive experiments on three public benchmarks demonstrate that our approach significantly outperforms prior methods in both fidelity and controllability. Our project page is at <a href="https://gulucaptain.github.io/UniCam/">https://gulucaptain.github.io/UniCam/</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

UniCam: Taming Unified Diffusion Models in Noise Space for Camera-controllable Video Rendering

  • 期刊论文

  • 发表日期：

    2026年7月29日

查看全文
  • Haoyu Zhao,
  • Zuxuan Wu,
  • Yu-Gang Jiang

摘要

Controlling camera pose in video diffusion models is essential for novel-view video rendering, yet existing approaches struggle to achieve precise control. Methods that directly inject numerical camera parameters into the diffusion backbone often fail to capture subtle viewpoint variations and lead to structural distortions or visual artifacts. To overcome these limitations, we propose UniCam, a unified framework that introduces a temporally coherent stochastic representation, termed CameraNoise, warped from camera intrinsic and extrinsic parameters. Unlike conventional approaches, CameraNoise embeds camera poses directly into the noise space. This makes our approach independent of scene appearance while faithfully encoding camera motion. Specifically, we introduce a novel Geometry-guided Reprojection Flow along with a CameraNoise warping algorithm, which jointly preserves the Gaussian prior of diffusion and ensures consistent noise propagation under camera transformations. By integrating CameraNoise into the diffusion process, the UniCam framework produces stable, high-quality videos with precise camera control across text-to-video, image-to-video, and video-to-video generation tasks. Extensive experiments on three public benchmarks demonstrate that our approach significantly outperforms prior methods in both fidelity and controllability. Our project page is at https://gulucaptain.github.io/UniCam/.