Convolutional neural network (CNN) performs well in several research fields and achieves high accuracy. With the development of CNNs, various hardware acceleration methods have been proposed. Instruction Set Architecture (ISA) based acceleration methods are gaining attention by providing flexibility and efficiency. Meanwhile, for mobile or embedded environments, there is a preference for variable precision quantization data to meet different accuracy or performance requirements. In this paper, we design a new RISC-V ISA extension, named RV-CVP, which supports variable precision specifically for CNNs. It has 3 computational instructions for CNNs and 3 memory access instructions that closely match the computation. We implement the RV-CVP extension on the open-source processor NutShell. Then a CNN programming library is developed, which allows flexible use of the RV-CVP instructions and building CNN models. Finally, we implement our design in simulation and FPGA platforms and evaluate the code density, performance, hardware resource usage, and power consumption. The results show that the RV-CVP extension improves instruction density by up to 1.678x and program performance by up to 42.06x at the single operator level. While on real CNN models, the RV-CVP extension improves performance by 3.8x–4.4x. The code is available at https://gitee.com/OpenBPU/RV-CVP .