This paper presents a knowledge distillation framework for pneumonia classification using a chest X-ray images dataset. We explore using deep learning models, including DenseNet121, MobileNetV2, and a novel Visual Knowledge Distillation (VKD) method, to improve the performance of a simplified student network as well as tackle the problem of interpretability. The teacher model, with five convolutional blocks, is distilled into a smaller student model with three convolutional blocks, aiming to maintain high accuracy while reducing complexity. Our results demonstrate that the VKD model achieves the highest accuracy (92%) and specificity (88%), outperforming both the teacher and student models. Furthermore, Grad-CAM visualizations highlight the model’s ability to focus on clinically relevant regions in X-ray images, improving interpretability. The extensive experiments demonstrate the effectiveness of the proposed framework, making it suitable for real-world pneumonia detection tasks.