Benchmarking Deep Learning Inference on RISC-V CPUs
摘要
Deep learning methods are widely used to solve problems in different areas, ranging from daily tasks on smartphones to complex scientific and engineering simulations. Most of the time spent on preparing deep models goes into labeling datasets and training neural networks, but the inference performance has a significant impact on the potential use of these models in various scenarios. Therefore, it is important to study this aspect. In this paper, we examine the performance of deep model inference on RISC-V processors. Specifically, we compare the capabilities of three frameworks: OpenVINO, TensorFlow Lite, and Apache TVM, in terms of optimizing deep model inference on these processors. Our study shows that TensorFlow Lite outperformed TVM in terms of inference speed on DenseNet-121 and GoogleNet-v4, with an average improvement of 2.69 and 2.47 times, respectively. The OpenVINO toolkit demonstrates excellent performance on x86 devices, but it is significantly inferior on RISC-V because of the lack of RISC-V-specific optimizations for the main computational kernels. The results show that deep learning frameworks can be already successfully used on RISC-V devices.