Image inpainting is the task that addresses the challenge of restoring missing or corrupted areas within digital images. This field is of utmost importance for maintaining integrity of visual media in scenarios where data corruption due to loss is unavoidable. This paper proposes Squeeze and Excite enhanced Vision Transformer GAN (SEViT-GAN), a state of the art model architecture for image inpainting that combines components from Vision Transformers with Residual Squeeze-and-Excitation blocks within a Generative Adversarial framework. SEViT-GAN addresses the challenge of reconstruction of images that are corrupted with missing pixels, to synthesize visually consistent and contextually meaningful generations without noticeable distortions. SEViT-GAN also incorporates an advanced dual discriminator architecture consisting of a patch-based discriminator along with a global discriminator, that further enhances the reconstruction with a powerful adversarial training methodology. This incorporation of SE blocks with a Vision Transformer tailored to image inpainting is the first of its kind. Intensive evaluations based on advanced metrics reveal that SEViT-GAN significantly outperforms models, including a ResNet17 based model, a VGG16-based feature extractor model, and a conventional Autoencoder Generative Adversarial Network architecture. The evaluation was performed across multiple standard metrics, with SEViT-GAN achieving a Structural Similarity Index of 0.91, Peak Signal-to-Noise Ratio of 30.4 decibels, Learned Perceptual Image Patch Similarity of 0.15, and Fréchet Inception Distance of 4.8. In addition, we evaluated performance using further advanced perceptual metrics such as Structural Content Loss, Edge Score, Haar Wavelet-based Perceptual Similarity Index, and Diversity. These results highlight the superior ability of SEViT-GAN in generating exceptional images surpassing other models and essentially redefining the limits of image inpainting technology.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

SEViT-GAN: Attention-Based GAN with SE Residual Blocks for Image Inpainting

  • K. R. Amarnath,
  • Nandini Nayakudi,
  • Lekha S. Nair

摘要

Image inpainting is the task that addresses the challenge of restoring missing or corrupted areas within digital images. This field is of utmost importance for maintaining integrity of visual media in scenarios where data corruption due to loss is unavoidable. This paper proposes Squeeze and Excite enhanced Vision Transformer GAN (SEViT-GAN), a state of the art model architecture for image inpainting that combines components from Vision Transformers with Residual Squeeze-and-Excitation blocks within a Generative Adversarial framework. SEViT-GAN addresses the challenge of reconstruction of images that are corrupted with missing pixels, to synthesize visually consistent and contextually meaningful generations without noticeable distortions. SEViT-GAN also incorporates an advanced dual discriminator architecture consisting of a patch-based discriminator along with a global discriminator, that further enhances the reconstruction with a powerful adversarial training methodology. This incorporation of SE blocks with a Vision Transformer tailored to image inpainting is the first of its kind. Intensive evaluations based on advanced metrics reveal that SEViT-GAN significantly outperforms models, including a ResNet17 based model, a VGG16-based feature extractor model, and a conventional Autoencoder Generative Adversarial Network architecture. The evaluation was performed across multiple standard metrics, with SEViT-GAN achieving a Structural Similarity Index of 0.91, Peak Signal-to-Noise Ratio of 30.4 decibels, Learned Perceptual Image Patch Similarity of 0.15, and Fréchet Inception Distance of 4.8. In addition, we evaluated performance using further advanced perceptual metrics such as Structural Content Loss, Edge Score, Haar Wavelet-based Perceptual Similarity Index, and Diversity. These results highlight the superior ability of SEViT-GAN in generating exceptional images surpassing other models and essentially redefining the limits of image inpainting technology.