Scholarlink

<p>Diffusion models have achieved remarkable success in text-to-image generation by producing high-fidelity, diverse images from textual descriptions. However, effectively controlling these models to meet fine-grained user intent remains a significant challenge. In this work, we propose a novel controllable generation framework based on noise-level supervision, termed <b>MHDAUNet</b>. Our method introduces a dual-path DDIM inversion strategy to inject semantic information directly into the noise space, enhancing alignment with textual prompts. Additionally, we design a multi-head differential attention U-Net architecture that improves the network’s ability to capture fine-grained visual details while maintaining computational efficiency. We conduct extensive experiments on five benchmarks–Pick-a-Pic, HPD v2, DrawBench, GenEval, and PartiPrompts–and validate our approach on three popular diffusion models: Stable Diffusion v1.5, PixArt-<InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(\alpha \)</EquationSource> </InlineEquation>, and Hunyuan-DiT. Results demonstrate that MHDAUNet consistently outperforms existing baselines across various metrics, including CLIP Score, PickScore, HPS v2, and aesthetic measures. Ablation studies further verify the contribution of each module and confirm the method’s strong cross-model generalization ability. Our work provides a new perspective on optimizing diffusion-based generation through semantically aligned noise guidance.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Mhdaunet: enhancing semantic consistency in diffusion models via dual-path noise alignment

  • 期刊论文

  • 发表日期：

    2026年2月13日

查看全文
  • Yijun Bei,
  • Hao Lv,
  • Sicheng Zuo,
  • Nianshu Wang,
  • Qinqin Chen

摘要

Diffusion models have achieved remarkable success in text-to-image generation by producing high-fidelity, diverse images from textual descriptions. However, effectively controlling these models to meet fine-grained user intent remains a significant challenge. In this work, we propose a novel controllable generation framework based on noise-level supervision, termed MHDAUNet. Our method introduces a dual-path DDIM inversion strategy to inject semantic information directly into the noise space, enhancing alignment with textual prompts. Additionally, we design a multi-head differential attention U-Net architecture that improves the network’s ability to capture fine-grained visual details while maintaining computational efficiency. We conduct extensive experiments on five benchmarks–Pick-a-Pic, HPD v2, DrawBench, GenEval, and PartiPrompts–and validate our approach on three popular diffusion models: Stable Diffusion v1.5, PixArt- \(\alpha \) , and Hunyuan-DiT. Results demonstrate that MHDAUNet consistently outperforms existing baselines across various metrics, including CLIP Score, PickScore, HPS v2, and aesthetic measures. Ablation studies further verify the contribution of each module and confirm the method’s strong cross-model generalization ability. Our work provides a new perspective on optimizing diffusion-based generation through semantically aligned noise guidance.