Consistency models are a new family of generative models that can achieve high sample quality in one- and few-step generation. Its solid theoretical framework establishes it as the foundation for subsequent studies. Currently, a consistency model can be distilled from a pre-trained diffusion model through consistency distillation (CD). The utility of pre-trained models endows CD with a fast training convergence speed, thereby saving computational resources given the access to numerous pre-trained models in the thriving diffusion model community. However, current CD performs worse in sample quality than consistency training (CT) with improved techniques. To match the sample quality of CT while retaining CD’s ability to leverage pre-trained models, we present a novel distillation method dubbed improved consistency distillation (iCD). Specifically, our iCD integrate the unbiased Monte Carlo estimator used in CT with ordinary differential equation coupled data-noise pairs derived from the pre-trained diffusion model. Our insightful theoretical analysis indicates that this integration can simplify the trajectories that the consistency model learns. Benefiting from the simplified trajectories, our iCD can improve the sample quality of CD. Extensive experiments demonstrate that iCD inherits the fast training convergence of CD, while maintaining or even improving the sample quality of CT.