<p>Knowledge distillation is a powerful technique to transfer knowledge from a large, high-capacity network to a smaller network to meet low-memory or fast-execution requirements. While conventional knowledge distillation effectively conveys knowledge through the Kullback-Leibler (KL) divergence, there remains uncertainty in the transmitted knowledge due to its lack of differentiation. We propose a novel method, Extended Knowledge Distillation (EKD), that systematically groups similar knowledge relationships between teacher and student using an unsupervised clustering approach, assigning varying transfer rates to each cluster to enhance knowledge transfer. First, we combine the error rate of the student with the conventional KL divergence to construct a two-dimensional knowledge space. Second, we perform clustering to group similar pieces of knowledge into subgroups. Third, we develop an algorithm that applies distinct transfer rates based on the characteristics of each cluster. This approach increases the efficiency of knowledge transfer, thereby improving performance, accelerating the learning process, preventing overfitting, and achieving effective regularization. To show the effectiveness of our approach for compression problem, we demonstrate the accuracy through performing experiments on CIFAR100 and Tiny ImageNet data for ResNet and MobileNet. We compare our results with those of existing knowledge distillation methods and obtain superior results.</p>