Running large neural networksNeural network on phones, IoT devices, and embedded systemsEmbedembedded system is difficult because of limited memoryMemory, storage, compute, and battery. Neural networkNeural network  compressionCompression methods aim to shrink models while keeping accuracy acceptable. This chapter surveys key methods for compressing neural networksNeural network, including pruningPruning, weight sharingWeightweight sharing, matrixMatrix decompositionDecompositionmatrix decomposition, and quantizationQuantization, with a particular focus onKnowledgeknowledge distillation knowledge distillationDistillation as a powerful approach for achieving efficient model compressionCompression. By presenting foundational concepts and recent advances, this chapter aims to provide a comprehensive understanding of how neural networksNeural network can be effectively compressed for deployment in resource-limited environments.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Neural Network Compression and Knowledge Distillation

  • Benyamin Ghojogh,
  • Ali Ghodsi

摘要

Running large neural networksNeural network on phones, IoT devices, and embedded systemsEmbedembedded system is difficult because of limited memoryMemory, storage, compute, and battery. Neural networkNeural network  compressionCompression methods aim to shrink models while keeping accuracy acceptable. This chapter surveys key methods for compressing neural networksNeural network, including pruningPruning, weight sharingWeightweight sharing, matrixMatrix decompositionDecompositionmatrix decomposition, and quantizationQuantization, with a particular focus onKnowledgeknowledge distillation knowledge distillationDistillation as a powerful approach for achieving efficient model compressionCompression. By presenting foundational concepts and recent advances, this chapter aims to provide a comprehensive understanding of how neural networksNeural network can be effectively compressed for deployment in resource-limited environments.