Video deblurring faces significant challenges due to complex blur patterns arising from both camera motion and object movement. While existing methods predominantly rely on distortion-based metrics like PSNR for evaluation, these measurements often show limited correlation with human visual perception and tend to produce unrealistic reconstructions. Recent advancements in diffusion models have demonstrated exceptional capabilities in generating realistic visual content, with image diffusion models achieving photorealistic synthesis and video diffusion models showing promise in temporal coherence. In this paper, we propose DIVD: Deblurring with Improved Video Diffusion Model, specifically designed for deblurring tasks. Our framework introduces two key enhancements: 1) Window-based Temporal Self-Attention (WTSA) for leveraging inter-frame dependencies. 2) Multi-frame Relative Positional Encoding (MRPE) for handling inter-frame inconsistencies. Extensive experiments demonstrate our model’s state-of-the-art performance across multiple perceptual metrics while maintaining competitive distortion metrics. Qualitative evaluations reveal significantly improved detail preservation compared to existing approaches. This work presents successful adaptation of diffusion models for video deblurring, effectively addressing the realism limitations of conventional approaches.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Deblurring with Improved Video Diffusion Model

  • Haoyang Long,
  • Bo Zhang,
  • Wufan Wang,
  • Zheng Zhang,
  • Wendong Wang

摘要

Video deblurring faces significant challenges due to complex blur patterns arising from both camera motion and object movement. While existing methods predominantly rely on distortion-based metrics like PSNR for evaluation, these measurements often show limited correlation with human visual perception and tend to produce unrealistic reconstructions. Recent advancements in diffusion models have demonstrated exceptional capabilities in generating realistic visual content, with image diffusion models achieving photorealistic synthesis and video diffusion models showing promise in temporal coherence. In this paper, we propose DIVD: Deblurring with Improved Video Diffusion Model, specifically designed for deblurring tasks. Our framework introduces two key enhancements: 1) Window-based Temporal Self-Attention (WTSA) for leveraging inter-frame dependencies. 2) Multi-frame Relative Positional Encoding (MRPE) for handling inter-frame inconsistencies. Extensive experiments demonstrate our model’s state-of-the-art performance across multiple perceptual metrics while maintaining competitive distortion metrics. Qualitative evaluations reveal significantly improved detail preservation compared to existing approaches. This work presents successful adaptation of diffusion models for video deblurring, effectively addressing the realism limitations of conventional approaches.