Neural networks have increasingly been adopted in practical applications, leading to a growing need for improving their computational efficiency and energy consumption. Pruning has emerged as a key technique in addressing this challenge. Traditional pruning methods typically involve training a dense neural network to completion, followed by the removal of less significant weights or neurons. However, this approach often results in suboptimal performance due to a mismatch between the training and inference phases. In this paper, we propose a pruning-aware training method based on regularization using the tent (triangular) function. Our method enables the neural network to jointly optimize for both sparsity and performance during training. We evaluate our approach on benchmark datasets, including MNIST, CIFAR-10, and CIFAR-100, demonstrating its effectiveness and superior performance compared to traditional pruning techniques.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Triangle Regularization for Pruning Aware Training of Neural Networks

  • Abdelfattah Toulaoui,
  • Mourad Jabrane,
  • Hamza Khalfi,
  • Imad Hafidi

摘要

Neural networks have increasingly been adopted in practical applications, leading to a growing need for improving their computational efficiency and energy consumption. Pruning has emerged as a key technique in addressing this challenge. Traditional pruning methods typically involve training a dense neural network to completion, followed by the removal of less significant weights or neurons. However, this approach often results in suboptimal performance due to a mismatch between the training and inference phases. In this paper, we propose a pruning-aware training method based on regularization using the tent (triangular) function. Our method enables the neural network to jointly optimize for both sparsity and performance during training. We evaluate our approach on benchmark datasets, including MNIST, CIFAR-10, and CIFAR-100, demonstrating its effectiveness and superior performance compared to traditional pruning techniques.