<p>Diffusion models have gained significant attention for their ability to generate high-quality images. However, most research focuses on image generation, overlooking the potential of conditional density estimation-a capability vital for other tasks. Recently, diffusion models have been applied to classification tasks, but challenges such as slow generation times and low prediction accuracy remain. To address these issues, we propose the CLIP Filter Diffusion Classifier (CFDC), a novel framework for efficient zero-shot image classification. CFDC introduces a Filter module that optimizes the integration of CLIP and diffusion models, effectively leveraging both image and text features. Experimental results show that CFDC more than doubles the classification speed compared to other diffusion models. It also achieves at least a 4% improvement in accuracy compared to existing zero-shot models such as CLIP on the MNIST dataset. These results demonstrate CFDC’s superior performance and underscore the potential of generative models for discriminative tasks. This marks a significant advancement in applying diffusion models to downstream applications.</p>