Inducing sparsity is an important way to improve the efficiency of deep neural networks. This chapter will discuss why sparsity is desired in a neural network model, and introduce regularization-based methods to induce such sparsity. We will start our discussion with the simple LASSO regularization, then move on to discuss the use of its variants and other regularization techniques beyond LASSO.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Regularization Techniques for Sparsity

  • Yiran Chen,
  • Hai Li,
  • Huanrui Yang

摘要

Inducing sparsity is an important way to improve the efficiency of deep neural networks. This chapter will discuss why sparsity is desired in a neural network model, and introduce regularization-based methods to induce such sparsity. We will start our discussion with the simple LASSO regularization, then move on to discuss the use of its variants and other regularization techniques beyond LASSO.