Diffusion models have recently gained widespread popularity in the field due to their exceptional image generation capabilities. Despite their powerful functionalities, the complex structure of these models and the step-by-step denoising process often lead to high computational costs and slow generation speeds, significantly limiting their wider application. Although various methods have been developed to reduce operational overhead and speed up image generation, these methods usually involve a trade-off between acceleration and maintaining quality. In this paper, we propose a new acceleration strategy that optimizes diffusion models by compressing the model structure and the generation process. Specifically, we first compress redundant tokens in the diffusion model’s generation process to reduce computational complexity. Next, we compress and reuse feature redundancies during the progressive sampling process to minimize unnecessary computation. To enhance the sampling efficiency of diffusion models, we employ an optimal path finding scheduler to approximate the entire generation process. We validated our method on a variety of datasets, including CIFAR, ImageNet and COCO2017, and tested under DDPM, LDM and Stable Diffusion.The experimental results confirmed the effectiveness of our approach in generating high-quality images across various settings of Stable Diffusion and LDM-4. Notably, we achieved a 2-6x acceleration effect while observing only minor changes in CLIP fraction.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multistage Compression Optimization Strategies for Accelerating Diffusion Models

  • Weiquan Huang,
  • Qiang Chen

摘要

Diffusion models have recently gained widespread popularity in the field due to their exceptional image generation capabilities. Despite their powerful functionalities, the complex structure of these models and the step-by-step denoising process often lead to high computational costs and slow generation speeds, significantly limiting their wider application. Although various methods have been developed to reduce operational overhead and speed up image generation, these methods usually involve a trade-off between acceleration and maintaining quality. In this paper, we propose a new acceleration strategy that optimizes diffusion models by compressing the model structure and the generation process. Specifically, we first compress redundant tokens in the diffusion model’s generation process to reduce computational complexity. Next, we compress and reuse feature redundancies during the progressive sampling process to minimize unnecessary computation. To enhance the sampling efficiency of diffusion models, we employ an optimal path finding scheduler to approximate the entire generation process. We validated our method on a variety of datasets, including CIFAR, ImageNet and COCO2017, and tested under DDPM, LDM and Stable Diffusion.The experimental results confirmed the effectiveness of our approach in generating high-quality images across various settings of Stable Diffusion and LDM-4. Notably, we achieved a 2-6x acceleration effect while observing only minor changes in CLIP fraction.