This paper presents the use of Paired Hierarchical Variational Autoencoders (PHVAEs) for image-to-image translation and cross reconstruction. We explore a novel method for constructing and applying HVAEs for these types of tasks, and compare this model against existing GAN, transformer, and diffusion based methods, including pix2pix, StegoGAN, ResViT, and BBDM. The method utilizes deep, hierarchical VAEs with paired optimization to take advantage of the bidirectional latent space. The proposed model outperforms all alternative models in PSNR on all tested tasks, and attains consistently competitive SSIM, demonstrating its effectiveness and benefit for prediction-focused translation tasks such as cross reconstruction, while remaining parameter efficient.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Paired Hierarchical VAEs for Image-to-Image Translation and Cross Reconstruction

  • Daniel Hess,
  • Tom Horrocks,
  • Daniel Wedge,
  • Eun-Jung Holden,
  • Ross P. Williams

摘要

This paper presents the use of Paired Hierarchical Variational Autoencoders (PHVAEs) for image-to-image translation and cross reconstruction. We explore a novel method for constructing and applying HVAEs for these types of tasks, and compare this model against existing GAN, transformer, and diffusion based methods, including pix2pix, StegoGAN, ResViT, and BBDM. The method utilizes deep, hierarchical VAEs with paired optimization to take advantage of the bidirectional latent space. The proposed model outperforms all alternative models in PSNR on all tested tasks, and attains consistently competitive SSIM, demonstrating its effectiveness and benefit for prediction-focused translation tasks such as cross reconstruction, while remaining parameter efficient.