Scholarlink

<p>Model compression can address the limitations of deep learning in resource-constrained situations by reducing the computational and storage requirements of the model. Structured pruning has emerged as an important compression technique because of its operational flexibility and effectiveness. However, the existing structural pruning methods have two limitations: 1) They use a single measurement to identify the importance of the filters in all the layers, resulting in a loss of spatial information in the shallow layers. 2) The pruning rate is highly dependent on manual interference, which is highly subjective. In this paper, a filter pruning method called <Emphasis Type="BoldItalic">d</Emphasis>ual <Emphasis Type="BoldItalic">a</Emphasis>ttention cooperative <Emphasis Type="BoldItalic">a</Emphasis>daptive pruning <Emphasis Type="BoldItalic">r</Emphasis>ate (DAAR) is proposed. Specifically, a dual attention module that combines spatial attention and channel attention is proposed to measure the effectiveness of the filters. Spatial attention is used in the shallow layers, and channel attention is used in the deep layers. This allows the filter measurements to consider spatial information effectively. An adaptive pruning rate adjustment strategy is also used to eliminate manual subjectivity, achieving precision pruning of each convolutional layer. The experimental results on various datasets and networks demonstrate that the DAAR method achieves improved model performance after pruning. For example, in the CIFAR10 dataset, the precision increases from 93.5% to 93.75% after removing the floating point operations (FLOPs) of 84.1%, outperforming the state-of-the-art pruning methods.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

DAAR: Dual attention cooperative adaptive pruning rate by data-driven for filter pruning

  • 期刊论文

  • 发表日期：

    2025年2月4日

查看全文
  • Suyun Lian,
  • Yang Zhao,
  • Jihong Pei

摘要

Model compression can address the limitations of deep learning in resource-constrained situations by reducing the computational and storage requirements of the model. Structured pruning has emerged as an important compression technique because of its operational flexibility and effectiveness. However, the existing structural pruning methods have two limitations: 1) They use a single measurement to identify the importance of the filters in all the layers, resulting in a loss of spatial information in the shallow layers. 2) The pruning rate is highly dependent on manual interference, which is highly subjective. In this paper, a filter pruning method called dual attention cooperative adaptive pruning rate (DAAR) is proposed. Specifically, a dual attention module that combines spatial attention and channel attention is proposed to measure the effectiveness of the filters. Spatial attention is used in the shallow layers, and channel attention is used in the deep layers. This allows the filter measurements to consider spatial information effectively. An adaptive pruning rate adjustment strategy is also used to eliminate manual subjectivity, achieving precision pruning of each convolutional layer. The experimental results on various datasets and networks demonstrate that the DAAR method achieves improved model performance after pruning. For example, in the CIFAR10 dataset, the precision increases from 93.5% to 93.75% after removing the floating point operations (FLOPs) of 84.1%, outperforming the state-of-the-art pruning methods.