Deep neural networks (DNNs) have achieved remarkable success in various applications. However, their deployment on edge devices is constrained by high computational and storage requirements. Quantizing neural networks to reduce model size has shown notable success in achieving low bit-width representation of parameters while maintaining the original network’s performance. Nonetheless, balancing model compression and performance remains challenging, especially with mixed precision approaches that use different bit-widths for different layers. Our method addresses this by effectively estimating layer importance through statistical measures computed on a default 8-bit quantized model. Layers are ranked based on their importance, and we adaptively select the bit-width precision for each layer, ensuring the quantized model’s accuracy remains within an acceptable threshold margin. This threshold is adaptively chosen for each layer based on its importance. Unlike previous mixed-precision methods that are difficult to tune and depend on costly optimization or search techniques, our approach is interpretable, efficient, and effective in adaptive bit-width selection. We apply the proposed network quantization method to the image classification task. Experimental results demonstrate the effectiveness of the proposed method on various DNN architectures.