Diffusion models have recently gained widespread adoption in visual generation. However, their iterative denoising process is computationally intensive, making real-time inference on embedded devices with limited power highly challenging. As a result, accelerating diffusion models has become a critical research focus. While existing acceleration techniques are primarily designed for UNet-based architectures, they are not directly applicable to Transformer-based diffusion models (DiT). A natural approach to speed up DiT is to skip certain blocks, but this often leads to significant degradation in generation quality. To address the unique challenges of DiT, we propose \(\Delta \) -Cache, a novel caching method that captures and stores the incremental changes between different blocks, effectively reducing computational costs while maintaining fidelity to the original output. Furthermore, we conduct a quantitative analysis of the relationship between DiT block depth and image generation quality. Our findings reveal that shallow DiT blocks primarily define global structures such as composition and outlines, while deeper blocks focus on refining details, with middle blocks playing an intermediate role. Building on these insights, we introduce a denoising property alignment method that adaptively skips computations for different blocks at various timesteps while preserving performance. Comprehensive experiments on PIXART- \(\alpha \) , SD3, and DiT-XL demonstrate that \(\Delta \) -DiT achieves a 1.6 \(\times \) speedup while even enhancing generation quality. Additionally, on the NVIDIA Jetson AGX Orin, our method delivers a 2.03 \(\times \) real-world inference speedup without sacrificing performance, highlighting its effectiveness in resource-constrained scenarios.
