<p>The popularization of multi-camera systems and multi-view image capture has led to the emergence of sparse multi-view image super-resolution (MVSR) as a promising research direction. The primary approach to sparse multi-view super-resolution (SR) currently involves extending traditional single-view SR and stereo SR frameworks, i.e., extracting features from each view and performing pixel-domain alignment and fusion to leverage cross-view reference information. However, this straightforward framework has two main drawbacks. First, performing cross-view fusion in the pixel domain disregards the spatial perception information that multi-view images provide. Second, feature alignment and fusion across views introduce considerable redundant and repetitive computations, which hinders further scalability to more viewpoints. This paper proposes a novel sparse multi-view SR framework based on a unified spatial representation reference. Specifically, the proposed method first computes a multi-plane image spatial representation from the multi-view images. This multi-plane image (MPI) representation encapsulates all the information from each view and has spatial perception. Subsequently, an upsampled reference image is rendered from the MPI representation for the low-resolution views. A high-low frequency separation fusion network is then proposed to upscale the input low-resolution images based on the rendered reference. Experimental results demonstrate the effectiveness of the proposed method for recovering high-frequency details.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Enhancing sparse multi-view super-resolution with unified multi-plane image spatial representation

  • Weiyi Liu,
  • Shanding Diao,
  • Zeyu Xiao,
  • Yuan Chen,
  • Wei Jia,
  • Yang Zhao

摘要

The popularization of multi-camera systems and multi-view image capture has led to the emergence of sparse multi-view image super-resolution (MVSR) as a promising research direction. The primary approach to sparse multi-view super-resolution (SR) currently involves extending traditional single-view SR and stereo SR frameworks, i.e., extracting features from each view and performing pixel-domain alignment and fusion to leverage cross-view reference information. However, this straightforward framework has two main drawbacks. First, performing cross-view fusion in the pixel domain disregards the spatial perception information that multi-view images provide. Second, feature alignment and fusion across views introduce considerable redundant and repetitive computations, which hinders further scalability to more viewpoints. This paper proposes a novel sparse multi-view SR framework based on a unified spatial representation reference. Specifically, the proposed method first computes a multi-plane image spatial representation from the multi-view images. This multi-plane image (MPI) representation encapsulates all the information from each view and has spatial perception. Subsequently, an upsampled reference image is rendered from the MPI representation for the low-resolution views. A high-low frequency separation fusion network is then proposed to upscale the input low-resolution images based on the rendered reference. Experimental results demonstrate the effectiveness of the proposed method for recovering high-frequency details.