Scholarlink

<p>Tensor decomposition can effectively reduce redundant parameters and FLOPs in deep neural network (DNN). DNNs with low parameters and computational complexity are more convenient to deploy on edge devices. As a combination of CANDECOMP/PARAFAC decomposition and Tucker decomposition, block term decomposition (BTD) demonstrates stronger generalization performance but presents challenges in configuring the number of Tucker tensors and the dimensions of each core tensor, making the tuning of decomposition rank more complex. To solve the problem of complex rank configuration in BTD, a novel block term decomposition of deep neural networks by variational Bayesian (BTD-VB) is proposed. BTD-VB mainly consists of three parts: selecting the rank of the network layer parameter tensor, compressing network parameters through BTD, and fine-tuning compressed model. The BTD-VB algorithm automatically estimates the rank of DNN parameters in input and output channel dimensions through the global analytic solution of empirical variational Bayesian matrix factorization. Extensive experimental results show that compared with state-of-the-art methods, BTD-VB achieves a higher compression rate while guaranteeing compression performance.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Block Term Decomposition of Deep Neural Networks by Variational Bayesian

  • 期刊论文

  • 发表日期：

    2025年12月1日

查看全文
  • Jie Liu,
  • Fan Yang,
  • Changhong Shi,
  • Cheng Wu,
  • Weirong Liu

摘要

Tensor decomposition can effectively reduce redundant parameters and FLOPs in deep neural network (DNN). DNNs with low parameters and computational complexity are more convenient to deploy on edge devices. As a combination of CANDECOMP/PARAFAC decomposition and Tucker decomposition, block term decomposition (BTD) demonstrates stronger generalization performance but presents challenges in configuring the number of Tucker tensors and the dimensions of each core tensor, making the tuning of decomposition rank more complex. To solve the problem of complex rank configuration in BTD, a novel block term decomposition of deep neural networks by variational Bayesian (BTD-VB) is proposed. BTD-VB mainly consists of three parts: selecting the rank of the network layer parameter tensor, compressing network parameters through BTD, and fine-tuning compressed model. The BTD-VB algorithm automatically estimates the rank of DNN parameters in input and output channel dimensions through the global analytic solution of empirical variational Bayesian matrix factorization. Extensive experimental results show that compared with state-of-the-art methods, BTD-VB achieves a higher compression rate while guaranteeing compression performance.