Scholarlink

<p>Personalized text-to-image diffusion has emerged as a promising approach for controlling generation through instance-level image inputs, yet it often suffers from high storage costs and overfitting due to redundancy in the fine-tuning process. In this work, we identify two types of redundancy–parameter redundancy and feature redundancy–and provide an in-depth quantitative and qualitative analysis of their roles. To mitigate parameter redundancy, we propose an efficient, adaptive fine-tuning strategy that applies singular value decomposition to the difference matrix between pre-trained and fine-tuned parameters, prunes low-rank components while maintaining the quality of generation. For feature redundancy, we develop a contrastive learning–based refocusing scheme, which leverages optimal transport to isolate essential subject-specific features and aligns text embeddings with these refined visual representations. Experimental results indicate that our method not only alleviates overfitting but also reduces storage demands. Furthermore, it generalizes effectively to multi-subject scenarios, offering a more stable and coherent personalized text-to-image diffusion framework.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Rethinking Personalized T2I Diffusion Models from the Perspective of Redundancy

  • 期刊论文

  • 发表日期：

    2026年4月7日

查看全文
  • Xierui Wang,
  • Bohan Lei,
  • Xiaoyin Xu,
  • Fei Wu,
  • Min Zhang

摘要

Personalized text-to-image diffusion has emerged as a promising approach for controlling generation through instance-level image inputs, yet it often suffers from high storage costs and overfitting due to redundancy in the fine-tuning process. In this work, we identify two types of redundancy–parameter redundancy and feature redundancy–and provide an in-depth quantitative and qualitative analysis of their roles. To mitigate parameter redundancy, we propose an efficient, adaptive fine-tuning strategy that applies singular value decomposition to the difference matrix between pre-trained and fine-tuned parameters, prunes low-rank components while maintaining the quality of generation. For feature redundancy, we develop a contrastive learning–based refocusing scheme, which leverages optimal transport to isolate essential subject-specific features and aligns text embeddings with these refined visual representations. Experimental results indicate that our method not only alleviates overfitting but also reduces storage demands. Furthermore, it generalizes effectively to multi-subject scenarios, offering a more stable and coherent personalized text-to-image diffusion framework.