In recent years, Dynamic Neural Networks (DNNs) have attracted significant attention as they can adapt to limited-resource platforms such as mobile devices and self-driving cars. DNNs can dynamically adjust their parameters and architectures through adaptive inference, reducing computational complexity and balancing accuracy and efficiency. This study presents the Group-Level Dynamic Neural Network (GLDNN), aiming to address the deployment challenges of DNNs in real-world scenarios. During training, this model constructs multiple subnets, each optimized for a different accuracy-efficiency trade-off. At deployment, the most suitable subnets can be selected according to the requirements of the target device, enabling efficient resource utilization and excellent performance. The experimental results on the FMNIST dataset are remarkable. Subnets A and B of the proposed model outperform the existing models by 50% and 70% respectively. Moreover, knowledge distillation training is employed to mitigate the accuracy decline caused by a high pruning rate, strongly demonstrating the effectiveness of the model.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

GLDNN: A Group-Level Dynamic Neural Network via Knowledge Distillation for Image Recognition

  • Yuhang Xiao,
  • Qi Zhao,
  • Ahmed Oluwatoyin,
  • Shuchang Lyu,
  • Alhassan Kamara

摘要

In recent years, Dynamic Neural Networks (DNNs) have attracted significant attention as they can adapt to limited-resource platforms such as mobile devices and self-driving cars. DNNs can dynamically adjust their parameters and architectures through adaptive inference, reducing computational complexity and balancing accuracy and efficiency. This study presents the Group-Level Dynamic Neural Network (GLDNN), aiming to address the deployment challenges of DNNs in real-world scenarios. During training, this model constructs multiple subnets, each optimized for a different accuracy-efficiency trade-off. At deployment, the most suitable subnets can be selected according to the requirements of the target device, enabling efficient resource utilization and excellent performance. The experimental results on the FMNIST dataset are remarkable. Subnets A and B of the proposed model outperform the existing models by 50% and 70% respectively. Moreover, knowledge distillation training is employed to mitigate the accuracy decline caused by a high pruning rate, strongly demonstrating the effectiveness of the model.