This paper proposes SkipNZ, a novel approach to reduce computational demands with negligible accuracy loss in the CNN inference processing. SkipNZ extends existing zero-value skipping technique and enables the skipping of unnecessary multiplications. The main idea is to filter out non-zero values if the exponent difference is large enough, so that unnecessary multiplications are skipped. The evaluation results show that the proposed technique significantly reduces the number of multiplications with negligible accuracy loss. Compared to the baseline, SkipNZ with Gap9 reduces execution time to 0.71 \(\times \) in AlexNet with 0.1% accuracy loss. In VGG16, SkipNZ with Gap8 lowers the execution time to 0.78 \(\times \) with no accuracy loss. Synthesis results confirm the practicality of the proposed approach, showing that the area and power consumption overheads of SkipNZ are only 0.5% and 0.1%, respectively, compared to the baseline.