<p>Text-to-image diffusion models have achieved remarkable success, highlighting the critical necessity of subject-driven generation. However, existing methods still struggle to balance subject fidelity with text controllability and fail to extract comprehensive representations from limited reference data. To address these limitations, we propose the Efficient Feature Extraction of Finetuning-Free Model (EFE-SDG). Specifically, it comprises three core components: the MOCI module employs large-scale models to augment reference information; the ASCA module utilizes a cross-attention decoupling strategy to extract high-level subject semantics; and the Subject Feature Adaptive Attention Rules dynamically fuse these features with low-level structural details extracted by a Ref-Diffusion branch to prevent distribution mismatches. Comparative experiments demonstrate that EFE-SDG achieves superior subject fidelity and competitive text alignment compared to existing advanced methods. Notably, our approach significantly improves data efficiency, requiring a substantially smaller dataset and fewer computational resources during training. Our code will be available at: <a href="https://gitee.com/yongzhenke/efe-sdg">https://gitee.com/yongzhenke/efe-sdg</a></p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

EFE-SDG: efficient feature extraction of finetuning-free model in subject-driven generation

  • Hao Li,
  • Yongzhen Ke,
  • Shuai Yang,
  • Kai Wang,
  • Yemeng Wu

摘要

Text-to-image diffusion models have achieved remarkable success, highlighting the critical necessity of subject-driven generation. However, existing methods still struggle to balance subject fidelity with text controllability and fail to extract comprehensive representations from limited reference data. To address these limitations, we propose the Efficient Feature Extraction of Finetuning-Free Model (EFE-SDG). Specifically, it comprises three core components: the MOCI module employs large-scale models to augment reference information; the ASCA module utilizes a cross-attention decoupling strategy to extract high-level subject semantics; and the Subject Feature Adaptive Attention Rules dynamically fuse these features with low-level structural details extracted by a Ref-Diffusion branch to prevent distribution mismatches. Comparative experiments demonstrate that EFE-SDG achieves superior subject fidelity and competitive text alignment compared to existing advanced methods. Notably, our approach significantly improves data efficiency, requiring a substantially smaller dataset and fewer computational resources during training. Our code will be available at: https://gitee.com/yongzhenke/efe-sdg