Recent advances in GANs and diffusion models have enhanced capabilities for low-level vision tasks like image generation and image-to-image translation, yet their computational intensity impedes deployment on resource-limited devices. While KD shows promise in high-level vision tasks, adapting it to pixel-wise generative tasks remains challenging. This chapter establishes a framework addressing these fundamental limitations through distillation mechanisms adapted to the unique characteristics of generative models in low-level vision tasks, aiming to bridge the efficiency-fidelity gap in real-world applications.