Scholarlink

<p>Most real-world applications require real-time response to accommodate specific application’s needs. Structured filter pruning algorithms for meeting real-time application demands have been widely proposed and have a significant improvement in this field of research. In this paper, we propose a novel approach based on generative adversarial networks (GAN) and define a new loss function for achieving better results in pruning filters in deep neural networks. We implement our proposed approach on the ResNet 56/110 and GoogLeNet architecture to verify the functionality of our novel algorithm. By leveraging High-Performance Computing (HPC) and parallel GPU processing during the adversarial training phase, MLKD compresses complex networks to enable strict real-time performance and low-latency inference on distributed edge computing devices. Our experiments on the CIFAR-10 dataset illustrate that the proposed approach reduces the number of floating-point operations (FLOPs) while maintaining the accuracy with an insignificant decrease in the accuracy. Accuracy reduction is 0.15%, 0.36%, and 0.08% for 57.6%, 49.8%, and 60.0% reduction in parameters for GoogLeNet, ResNet-56, and ResNet-110, respectively. Moreover, our NVIDIA Jetson Nano Board implementation confirms the reduction in the inference time and power consumption of 0.34 W, 0.8 W, and 0.7 W for GoogLeNet, ResNet-56, and ResNet-110, respectively. The proposed approach achieves 15.8%, 24.6%, and 17% speedup per power for ResNet-56, ResNet-110, and GoogLeNet, respectively.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

MLKD: model’s loss-aware knowledge distillation for pruning deep convolutional neural networks

  • 期刊论文

  • 发表日期：

    2026年6月30日

查看全文
  • Mahdi Shamisavi,
  • Bijan Alizadeh

摘要

Most real-world applications require real-time response to accommodate specific application’s needs. Structured filter pruning algorithms for meeting real-time application demands have been widely proposed and have a significant improvement in this field of research. In this paper, we propose a novel approach based on generative adversarial networks (GAN) and define a new loss function for achieving better results in pruning filters in deep neural networks. We implement our proposed approach on the ResNet 56/110 and GoogLeNet architecture to verify the functionality of our novel algorithm. By leveraging High-Performance Computing (HPC) and parallel GPU processing during the adversarial training phase, MLKD compresses complex networks to enable strict real-time performance and low-latency inference on distributed edge computing devices. Our experiments on the CIFAR-10 dataset illustrate that the proposed approach reduces the number of floating-point operations (FLOPs) while maintaining the accuracy with an insignificant decrease in the accuracy. Accuracy reduction is 0.15%, 0.36%, and 0.08% for 57.6%, 49.8%, and 60.0% reduction in parameters for GoogLeNet, ResNet-56, and ResNet-110, respectively. Moreover, our NVIDIA Jetson Nano Board implementation confirms the reduction in the inference time and power consumption of 0.34 W, 0.8 W, and 0.7 W for GoogLeNet, ResNet-56, and ResNet-110, respectively. The proposed approach achieves 15.8%, 24.6%, and 17% speedup per power for ResNet-56, ResNet-110, and GoogLeNet, respectively.