Multi-image super-resolution consists in reconstructing high-resolution images from multiple low-resolution inputs. Such techniques are particularly valuable when achieving high spatial resolution is costly or infeasible, like in many cases related with remote sensing based on satellite images. The state-of-the-art approaches are underpinned with deep neural networks which are commonly trained relying on loss functions penalizing pixel-wise dissimilarity between the reconstructed image and a high-resolution reference. However, such loss functions are of limited efficacy when a network is trained from real-world datasets with target images acquired with a different sensor than the low-resolution inputs. In this paper, we address this problem by introducing a new metric which is learned to capture the similarity in the domain of keypoint features. Subsequently, we exploit the proposed metric as a loss function for training a neural network that realizes multi-image super-resolution. We report the results of our extensive experimental study which demonstrate that the new loss function can be effectively combined with conventional pixel-wise similarity for real-world super-resolution of satellite images. The obtained models improve the reconstruction quality both quantitatively and qualitatively, as confirmed with a survey conducted among human experts. The source code is available at https://github.com/pawel-benecki-polsl/pricai-25 .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Metric Learning for Multi-image Super-Resolution

  • Pawel Benecki,
  • Daniel Kostrzewa,
  • Michal Kawulok

摘要

Multi-image super-resolution consists in reconstructing high-resolution images from multiple low-resolution inputs. Such techniques are particularly valuable when achieving high spatial resolution is costly or infeasible, like in many cases related with remote sensing based on satellite images. The state-of-the-art approaches are underpinned with deep neural networks which are commonly trained relying on loss functions penalizing pixel-wise dissimilarity between the reconstructed image and a high-resolution reference. However, such loss functions are of limited efficacy when a network is trained from real-world datasets with target images acquired with a different sensor than the low-resolution inputs. In this paper, we address this problem by introducing a new metric which is learned to capture the similarity in the domain of keypoint features. Subsequently, we exploit the proposed metric as a loss function for training a neural network that realizes multi-image super-resolution. We report the results of our extensive experimental study which demonstrate that the new loss function can be effectively combined with conventional pixel-wise similarity for real-world super-resolution of satellite images. The obtained models improve the reconstruction quality both quantitatively and qualitatively, as confirmed with a survey conducted among human experts. The source code is available at https://github.com/pawel-benecki-polsl/pricai-25 .