In communication systems, automatic modulation recognition facilitates the development of many critical signal processing applications, such as cognitive radio and spectrum sharing. However, the high computational cost and large model sizes pose significant obstacles for deploying traditional deep learning-based methods, especially in IoT networks and UAV-assisted systems. To resolve this matter, this paper proposes the method of knowledge distillation (KD), selecting the highly accurate Deep Residual Shrinkage Network (DRSN) as the teacher network and the lightweight ShuffleNetV2 as the student network. We specifically design a self-attention enhanced KD module: the self-attention layer is embedded after the 3rd convolutional block of both teacher and student networks, calculating attention weights via scaled dot-product attention to highlight critical features. The training process integrates an additional attention loss term (MSE between teacher and student attention weights) into the total loss function. Experiments on the RadioML2016.10A dataset indicate that the distilled network, while maintaining the low parameter count of the student network, achieves accuracy close to the teacher network.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Research on Automatic Modulation Recognition Method Based on Knowledge Distillation

  • Ruoyu Zhou,
  • Zhuoran Cai,
  • Guangda Xin

摘要

In communication systems, automatic modulation recognition facilitates the development of many critical signal processing applications, such as cognitive radio and spectrum sharing. However, the high computational cost and large model sizes pose significant obstacles for deploying traditional deep learning-based methods, especially in IoT networks and UAV-assisted systems. To resolve this matter, this paper proposes the method of knowledge distillation (KD), selecting the highly accurate Deep Residual Shrinkage Network (DRSN) as the teacher network and the lightweight ShuffleNetV2 as the student network. We specifically design a self-attention enhanced KD module: the self-attention layer is embedded after the 3rd convolutional block of both teacher and student networks, calculating attention weights via scaled dot-product attention to highlight critical features. The training process integrates an additional attention loss term (MSE between teacher and student attention weights) into the total loss function. Experiments on the RadioML2016.10A dataset indicate that the distilled network, while maintaining the low parameter count of the student network, achieves accuracy close to the teacher network.