Text-to-image diffusion models demonstrate remarkable capabilities in generating high-quality images. However, diffusion models struggle with generating the correct number of instances, leading to issues such as missing or excessive instances, subject fusion, and incorrect attribute binding. These issues can be attributed to incorrect instance numbers, indicating that generating the correct number of instances can effectively avoid them. To address these issues, this paper proposes a training-free method that enables diffusion models to generate images with the correct number of instances. The method automatically identifies non-overlapping instance regions during inference. These regions are then refined via constraint loss functions applied to cross-attention maps. The proposed method, named Auto-Locate, generalizes well to multi-subject and multi-instance scenarios, enabling diffusion models to better handle complex text prompts involving diverse entities. Extensive experiments show that Auto-Locate effectively controls the number of instances in generated images while maintaining high fidelity and diversity, outperforming several baselines on standard benchmarks.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Auto-Locate: A Training-Free Multi-instance Generation for Text-to-Image Diffusion Models

  • Xiangzhi Tao,
  • Kuangzhi Wang,
  • Zhongyang Hu,
  • Naijie Gu

摘要

Text-to-image diffusion models demonstrate remarkable capabilities in generating high-quality images. However, diffusion models struggle with generating the correct number of instances, leading to issues such as missing or excessive instances, subject fusion, and incorrect attribute binding. These issues can be attributed to incorrect instance numbers, indicating that generating the correct number of instances can effectively avoid them. To address these issues, this paper proposes a training-free method that enables diffusion models to generate images with the correct number of instances. The method automatically identifies non-overlapping instance regions during inference. These regions are then refined via constraint loss functions applied to cross-attention maps. The proposed method, named Auto-Locate, generalizes well to multi-subject and multi-instance scenarios, enabling diffusion models to better handle complex text prompts involving diverse entities. Extensive experiments show that Auto-Locate effectively controls the number of instances in generated images while maintaining high fidelity and diversity, outperforming several baselines on standard benchmarks.