Scholarlink

<p>In recent years, personalized text-to-image (T2I) methods based on diffusion models have rapidly advanced, enabling the generation of high-quality images guided by both text prompts and reference images. However, current approaches face two main limitations. First, they often struggle to perform fine-grained edits to facial attributes without affecting the overall facial structure or background. Second, the generated images may fail to accurately reflect the semantic intent of the prompts, resulting in unclear or incorrect attribute changes. To address these challenges in a unified way, FISA (Fusion of Identity and Structure through Attention) is introduced as a facial attribute editing framework that integrates three complementary mechanisms rather than a simple combination of existing techniques. The first augments each U-Net layer with identity-oriented cross-attention to inject features from the reference image, providing an explicit identity condition for high-fidelity preservation. The second records self-attention weights during reconstruction of the noisy reference image and reuses them in the editing phase to anchor the global facial structure. Additionally, a noise-direction-based self-distillation loss regularizes the difference of noise predictions, enhancing semantic alignment with the text prompt while preventing identity drift. Extensive experiments demonstrate that FISA achieves superior perceptual quality and strong quantitative performance, enabling precise facial attribute edits while maintaining stability in all other regions.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Identity and structure preserving face editing via diffusion models

  • 期刊论文

  • 发表日期：

    2026年2月3日

查看全文
  • Xuyuan Liu,
  • Xiping He,
  • Yi Li,
  • Xiaoqin Xie,
  • Yan Zhang

摘要

In recent years, personalized text-to-image (T2I) methods based on diffusion models have rapidly advanced, enabling the generation of high-quality images guided by both text prompts and reference images. However, current approaches face two main limitations. First, they often struggle to perform fine-grained edits to facial attributes without affecting the overall facial structure or background. Second, the generated images may fail to accurately reflect the semantic intent of the prompts, resulting in unclear or incorrect attribute changes. To address these challenges in a unified way, FISA (Fusion of Identity and Structure through Attention) is introduced as a facial attribute editing framework that integrates three complementary mechanisms rather than a simple combination of existing techniques. The first augments each U-Net layer with identity-oriented cross-attention to inject features from the reference image, providing an explicit identity condition for high-fidelity preservation. The second records self-attention weights during reconstruction of the noisy reference image and reuses them in the editing phase to anchor the global facial structure. Additionally, a noise-direction-based self-distillation loss regularizes the difference of noise predictions, enhancing semantic alignment with the text prompt while preventing identity drift. Extensive experiments demonstrate that FISA achieves superior perceptual quality and strong quantitative performance, enabling precise facial attribute edits while maintaining stability in all other regions.