Scholarlink

<p>Video generation has achieved remarkable progress with the adoption of diffusion models and transformer-based architectures, enabling the creation of highly realistic and intricate video content. However, the quadratic computational complexity of attention in diffusion transformers with respect to the video length poses a significant challenge, particularly for high-resolution and long-duration videos. Quantization has been widely used for optimizing inference. However, the quantization of attention layers in video diffusion models remains difficult owing to the sensitivity to quantization errors. To tackle these issues, we propose Shenbi, the first INT8 attention solution for video diffusion models. Shenbi employs fine-grained block quantization and softmax-friendly quantization to ensure high precision. It further employs rowmax fusion and scale factor fusion techniques to reduce de/quantization overhead and adopts a hybrid quantization and attention block size strategy to maximize GPU utilization. The experimental results show that Shenbi delivers a 2.63× throughput for attention kernels and a 1.58× speedup for the entire model compared to the FP16 baseline, while preserving high model accuracy. The source code is available at <a href="https://github.com/HaiShuangFan/Shenbi">https://github.com/HaiShuangFan/Shenbi</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Shenbi: accelerating video diffusion models with INT8-attention

  • 期刊论文

  • 发表日期：

    2026年4月23日

查看全文
  • Haishuang Fan,
  • Ao Lv,
  • Guihai Yan

摘要

Video generation has achieved remarkable progress with the adoption of diffusion models and transformer-based architectures, enabling the creation of highly realistic and intricate video content. However, the quadratic computational complexity of attention in diffusion transformers with respect to the video length poses a significant challenge, particularly for high-resolution and long-duration videos. Quantization has been widely used for optimizing inference. However, the quantization of attention layers in video diffusion models remains difficult owing to the sensitivity to quantization errors. To tackle these issues, we propose Shenbi, the first INT8 attention solution for video diffusion models. Shenbi employs fine-grained block quantization and softmax-friendly quantization to ensure high precision. It further employs rowmax fusion and scale factor fusion techniques to reduce de/quantization overhead and adopts a hybrid quantization and attention block size strategy to maximize GPU utilization. The experimental results show that Shenbi delivers a 2.63× throughput for attention kernels and a 1.58× speedup for the entire model compared to the FP16 baseline, while preserving high model accuracy. The source code is available at https://github.com/HaiShuangFan/Shenbi.