Sketch face synthesis aims to create detailed and realistic sketch images from optical photos. Recently, diffusion models have effectively addressed challenges like over-smoothing and mode collapse by simulating the distribution of multi-channel input data, but due to limitations in capturing low-frequency information, the generated images lack authenticity in textural details. To address the concerns raised in the appeal, we propose the Conditional Denoising Diffusion Probability Model (DDPM) with The T-UNet. To improve the utilization of low-frequency information, we propose a T-UNet module. This module maintains UNet’s ability to capture high-frequency details while integrating the Transformer’s ability to process low-frequency information. As a result, it enhances the overall quality of the synthesized sketch images. Experiments on the CUHK dataset demonstrate that our approach generates sketches of excellent quality, surpassing existing methods in visual performance.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Conditional Diffusion Model with T-UNet for Sketch Face Synthesis

  • Jiyu Zhang,
  • Kangning Du,
  • Huanyu Bian,
  • Pengcheng Wang

摘要

Sketch face synthesis aims to create detailed and realistic sketch images from optical photos. Recently, diffusion models have effectively addressed challenges like over-smoothing and mode collapse by simulating the distribution of multi-channel input data, but due to limitations in capturing low-frequency information, the generated images lack authenticity in textural details. To address the concerns raised in the appeal, we propose the Conditional Denoising Diffusion Probability Model (DDPM) with The T-UNet. To improve the utilization of low-frequency information, we propose a T-UNet module. This module maintains UNet’s ability to capture high-frequency details while integrating the Transformer’s ability to process low-frequency information. As a result, it enhances the overall quality of the synthesized sketch images. Experiments on the CUHK dataset demonstrate that our approach generates sketches of excellent quality, surpassing existing methods in visual performance.