Image classification is a fundamental task in computer vision, with convolutional neural networks (CNNs) being the state-of-the-art approach for this task. However, CNNs can be large and computationally expensive, which limits their deployment on resource-constrained devices. To address this issue, model optimization and compression techniques such as knowledge distillation and channel attention have been proposed. In this paper, we propose a modified version of the skip-connection knowledge distillation method, which integrates the efficient channel attention module. Our experiments on the CIFAR-10 dataset show that this approach improves the performance of both large teacher-small student and self-distillation scenarios, and also leads to smaller model sizes. We also visualize the intermediate feature maps to understand the learning process of the network. Our results demonstrate the effectiveness of combining knowledge distillation and channel attention for improving the performance and efficiency of CNNs.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Improving Image Classification Efficiency with Knowledge Distillation and Channel Attention

  • Youssef Boulaouane,
  • Jisu Kim,
  • Jimin Park,
  • Deokwoo Lee

摘要

Image classification is a fundamental task in computer vision, with convolutional neural networks (CNNs) being the state-of-the-art approach for this task. However, CNNs can be large and computationally expensive, which limits their deployment on resource-constrained devices. To address this issue, model optimization and compression techniques such as knowledge distillation and channel attention have been proposed. In this paper, we propose a modified version of the skip-connection knowledge distillation method, which integrates the efficient channel attention module. Our experiments on the CIFAR-10 dataset show that this approach improves the performance of both large teacher-small student and self-distillation scenarios, and also leads to smaller model sizes. We also visualize the intermediate feature maps to understand the learning process of the network. Our results demonstrate the effectiveness of combining knowledge distillation and channel attention for improving the performance and efficiency of CNNs.