<p>Model compression or lightweighting is a critical task in deep learning to reduce memory usage and improve processing speed. Pruning, quantization, and knowledge distillation have been individually studied as effective approaches. However, efforts to combine these techniques for synergistic performance improvements have been limited. In particular, due to data type changes introduced by quantization, it becomes difficult to apply pruning or distillation afterward, and most existing approaches are limited to determining only the order between pruning and distillation. In this paper, we first propose an integrated compression pipeline that effectively combines pruning, quantization, QAT (Quantization-Aware Training), and knowledge distillation, considering the trade-offs and synergistic efficiency among these processes. Second, we introduce an efficiency score defined as the harmonic mean between the model compression rate and accuracy drop, to provide a balanced evaluation between model size reduction and performance retention. Third, we propose a Compression-Aware Knowledge Distillation (CAKD) approach that eliminates the need for an external teacher model and employs a two-stage distillation process during both fine-tuning and QAT phases to achieve further performance improvement. We validate our method through experiments using the ResNet-56 and MobileNetV2 models on the CIFAR-100 and Tiny ImageNet datasets. The results show that our pipeline achieves a better trade-off between model compression and accuracy degradation compared to existing techniques.</p>