<p>Video Super-Resolution (VSR) is widely used in real-time rendering and video processing due to its ability to improve image quality with low computational costs. Existing methods reduce latency with the help of G-buffers, demonstrating certain advantages in lowering video latency and real-time super-resolution tasks. However, they perform poorly in handling the detailed textures of video frames.To address this issue,We first adopt an asymmetric super - resolution network based on U-Net with a decoupled G-buffer guiding mechanism to explore temporal and spatial features more efficiently and enhance video frame details, which first designs a Spatial Gating Filter (SGF) to selectively encode spatial information for accurate capture of image structural information. We then introduces auxiliary G-buffer information and designs an Adaptive Feature Modulator (AFM) to guide the decoder in generating detailed and temporally stable results by enabling adaptive transmission of high-frequency information and extracting high-frequency details. we also constructs Convolutional Block Attention Module - Convolutional Channel Mixer(CBAM-CCM) that enhances feature representation via the CBAM attention mechanism and adaptively adjusts feature responses in channel and spatial dimensions. The experimental results show that the parameters of our network are reduced to 100.51M, the latency is decreased to 18.21ms, and the Peak Signal-to-Noise Ratio(PSNR) is improved to 30.21dB. Our method outperforms existing algorithms in both accuracy and efficiency.Our source codes are available at: <a href="https://github.com/jiangguolong/Video-Super-Resolution-for-Real-Time-Rendering">https://github.com/jiangguolong/Video-Super-Resolution-for-Real-Time-Rendering</a></p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Video super-resolution for real-time rendering: decoupled G-buffer guidance with attention-enhanced modulation

  • Guolong Jiang,
  • Baoshu Xu,
  • Qunzhong Fang

摘要

Video Super-Resolution (VSR) is widely used in real-time rendering and video processing due to its ability to improve image quality with low computational costs. Existing methods reduce latency with the help of G-buffers, demonstrating certain advantages in lowering video latency and real-time super-resolution tasks. However, they perform poorly in handling the detailed textures of video frames.To address this issue,We first adopt an asymmetric super - resolution network based on U-Net with a decoupled G-buffer guiding mechanism to explore temporal and spatial features more efficiently and enhance video frame details, which first designs a Spatial Gating Filter (SGF) to selectively encode spatial information for accurate capture of image structural information. We then introduces auxiliary G-buffer information and designs an Adaptive Feature Modulator (AFM) to guide the decoder in generating detailed and temporally stable results by enabling adaptive transmission of high-frequency information and extracting high-frequency details. we also constructs Convolutional Block Attention Module - Convolutional Channel Mixer(CBAM-CCM) that enhances feature representation via the CBAM attention mechanism and adaptively adjusts feature responses in channel and spatial dimensions. The experimental results show that the parameters of our network are reduced to 100.51M, the latency is decreased to 18.21ms, and the Peak Signal-to-Noise Ratio(PSNR) is improved to 30.21dB. Our method outperforms existing algorithms in both accuracy and efficiency.Our source codes are available at: https://github.com/jiangguolong/Video-Super-Resolution-for-Real-Time-Rendering