<p>Face sketch and photo synthesis is widely applied in industry and information fields, such as entertainment business and heterogeneous face retrieval. The key challenge lies in completing a face transformation with both good visual effects and face identity preservation. However, existing methods are still difficult to obtain a good synthesis due to the large model gap between the two different face domains. Recently, diffusion models have achieved great success in image synthesis, which allows us to extend its application in such a face generation task. Thus, we propose IPDM, which constructs a mapping of latent representation for domain-adaptive face features. The other proposed IDP utilizes auxiliary features to correct the latent features through their directions and supplementary identity information, so that the generation can keep face identity unchanged. The various evaluation results show that our method is superior to state-of-the-art methods in both identity preservation and visual effects.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Ipdm: identity preserving diffusion model for face sketch and photo synthesis

  • Duoxun Tang,
  • Xinhang Jiang,
  • Ying Zhang,
  • Yuhang Dai,
  • Ye Lin

摘要

Face sketch and photo synthesis is widely applied in industry and information fields, such as entertainment business and heterogeneous face retrieval. The key challenge lies in completing a face transformation with both good visual effects and face identity preservation. However, existing methods are still difficult to obtain a good synthesis due to the large model gap between the two different face domains. Recently, diffusion models have achieved great success in image synthesis, which allows us to extend its application in such a face generation task. Thus, we propose IPDM, which constructs a mapping of latent representation for domain-adaptive face features. The other proposed IDP utilizes auxiliary features to correct the latent features through their directions and supplementary identity information, so that the generation can keep face identity unchanged. The various evaluation results show that our method is superior to state-of-the-art methods in both identity preservation and visual effects.