This paper presents a novel transformer-guided encoder-decoder network for facial Image translation by using multi-layer perceptron and various depth levels. The proposed network exploits the advantage of multi-head self-attention mechanisms for efficient learning. Inspired by the generative capabilities of the generative adversarial network (GAN) for image-to-image translation, we employed the proposed Transformer-Inception based Encoder in the GAN framework. The proposed encoder-decoder network is used as a generator network. The proposed TranscoderGAN model facilitates the intense cyclic training for image-to-image translation efficiently by utilizing inception-based learning in residual networks. The proposed TranscoderGAN is used for facial image synthesis over the sketch in the CUHK dataset as well as the thermal faces of the WHU-IIP dataset. The proposed TranscoderGAN outperforms the state-of-the-art GAN methods for facial image translation. In quantitative results analysis, on an average \(\{\) 10.36%, 0.29% \(\}\) improvement is found over Structural Similarity Index Measure (SSIM) and Visual Information Fidelity (VIF), respectively, while \(\{\) 55.38% \(\}\) reduction over the Fréchet Inception Distance (Fid) for the CUHK dataset. For WHU-IIP thermal face dataset, the average improvement by the proposed model is \(\{\) 9.63 \(\%, \) 1.14 \(\%\}\) over SSIM, VIF, and \(\{\) 89.25 \(\%\}\) reduction in Fid metrics, respectively.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

TranscoderGAN: Transformer-Inception Based Encoder-Decoder Generative Adversarial Network for Thermal-Visible Face Transformation

  • Nand Kumar Yadav,
  • Satish Kumar Singh,
  • Shiv Ram Dubey

摘要

This paper presents a novel transformer-guided encoder-decoder network for facial Image translation by using multi-layer perceptron and various depth levels. The proposed network exploits the advantage of multi-head self-attention mechanisms for efficient learning. Inspired by the generative capabilities of the generative adversarial network (GAN) for image-to-image translation, we employed the proposed Transformer-Inception based Encoder in the GAN framework. The proposed encoder-decoder network is used as a generator network. The proposed TranscoderGAN model facilitates the intense cyclic training for image-to-image translation efficiently by utilizing inception-based learning in residual networks. The proposed TranscoderGAN is used for facial image synthesis over the sketch in the CUHK dataset as well as the thermal faces of the WHU-IIP dataset. The proposed TranscoderGAN outperforms the state-of-the-art GAN methods for facial image translation. In quantitative results analysis, on an average \(\{\) 10.36%, 0.29% \(\}\) improvement is found over Structural Similarity Index Measure (SSIM) and Visual Information Fidelity (VIF), respectively, while \(\{\) 55.38% \(\}\) reduction over the Fréchet Inception Distance (Fid) for the CUHK dataset. For WHU-IIP thermal face dataset, the average improvement by the proposed model is \(\{\) 9.63 \(\%, \) 1.14 \(\%\}\) over SSIM, VIF, and \(\{\) 89.25 \(\%\}\) reduction in Fid metrics, respectively.