Deep learning methods are widely used to solve problems in different areas, ranging from daily tasks on smartphones to complex scientific and engineering simulations. Most of the time spent on preparing deep models goes into labeling datasets and training neural networks, but the inference performance has a significant impact on the potential use of these models in various scenarios. Therefore, it is important to study this aspect. In this paper, we examine the performance of deep model inference on RISC-V processors. Specifically, we compare the capabilities of three frameworks: OpenVINO, TensorFlow Lite, and Apache TVM, in terms of optimizing deep model inference on these processors. Our study shows that TensorFlow Lite outperformed TVM in terms of inference speed on DenseNet-121 and GoogleNet-v4, with an average improvement of 2.69 and 2.47 times, respectively. The OpenVINO toolkit demonstrates excellent performance on x86 devices, but it is significantly inferior on RISC-V because of the lack of RISC-V-specific optimizations for the main computational kernels. The results show that deep learning frameworks can be already successfully used on RISC-V devices.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Benchmarking Deep Learning Inference on RISC-V CPUs

  • Ivan Mukhin,
  • Yury Rodimkov,
  • Evgeny Vasiliev,
  • Valentin Volokitin,
  • Alexandra Sidorova,
  • Evgeny Kozinov,
  • Iosif Meyerov,
  • Valentina Kustikova

摘要

Deep learning methods are widely used to solve problems in different areas, ranging from daily tasks on smartphones to complex scientific and engineering simulations. Most of the time spent on preparing deep models goes into labeling datasets and training neural networks, but the inference performance has a significant impact on the potential use of these models in various scenarios. Therefore, it is important to study this aspect. In this paper, we examine the performance of deep model inference on RISC-V processors. Specifically, we compare the capabilities of three frameworks: OpenVINO, TensorFlow Lite, and Apache TVM, in terms of optimizing deep model inference on these processors. Our study shows that TensorFlow Lite outperformed TVM in terms of inference speed on DenseNet-121 and GoogleNet-v4, with an average improvement of 2.69 and 2.47 times, respectively. The OpenVINO toolkit demonstrates excellent performance on x86 devices, but it is significantly inferior on RISC-V because of the lack of RISC-V-specific optimizations for the main computational kernels. The results show that deep learning frameworks can be already successfully used on RISC-V devices.