Model quantization reduces latency, improves speed, and saves energy by making models smaller and less demanding to run. Post-Training Quantization (PTQ) is faster than Quantization-Aware Training (QAT) methods but often hurts accuracy when using very low precision. Mixed-precision quantization assigns different precision levels to different layers to balance accuracy and efficiency, but finding the best combination is difficult due to too many possible configurations and high evaluation costs. This paper introduces HEMQ (Hybrid Evolutionary Mixed-Precision Quantization), a new approach for PTQ that uses a fast error measurement method and combines genetic algorithms with differential evolution to efficiently search for optimal precision settings. It also uses a two-stage optimization process to improve accuracy. Experiments show that HEMQ outperforms existing methods across various tasks and network types, providing a practical solution for running high-performance models on devices with limited resources.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

HEMQ: Towards More Accurate Post-training Mixed-Precision Quantization of Neural Networks

  • Yong Yuan,
  • Hang Shi,
  • Yuan Gao

摘要

Model quantization reduces latency, improves speed, and saves energy by making models smaller and less demanding to run. Post-Training Quantization (PTQ) is faster than Quantization-Aware Training (QAT) methods but often hurts accuracy when using very low precision. Mixed-precision quantization assigns different precision levels to different layers to balance accuracy and efficiency, but finding the best combination is difficult due to too many possible configurations and high evaluation costs. This paper introduces HEMQ (Hybrid Evolutionary Mixed-Precision Quantization), a new approach for PTQ that uses a fast error measurement method and combines genetic algorithms with differential evolution to efficiently search for optimal precision settings. It also uses a two-stage optimization process to improve accuracy. Experiments show that HEMQ outperforms existing methods across various tasks and network types, providing a practical solution for running high-performance models on devices with limited resources.