Scholarlink

The RISC-V instruction set architecture is gaining increasing attention in embedded and edge computing due to its open source, AI scalability, and low power consumption, which make it well-suited for edge-side AI applications. However, the standard operator library is not optimized for RISC-V well, which results in suboptimal performance due to the underutilization of the AI scalability. Therefore, using the RISC-V Vector (RVV) extension, we design and implement a high-performance operator library to address this challenge and enhance operator computing efficiency. The library is compatible with most inference frameworks and different RVV versions. Meanwhile, it accelerates several operations, such as convolution, activation functions, and basic arithmetic, by converting scalar calculations into vector calculations through loop expansion. We compared the performance of our operator library across three types of applications: YOLOv8n, llama.cpp, and StableDiffusion. The results show that, compared to scalar implementation, our approach achieves a performance improvement of 32.09% to 63.97% in YOLOv8n, a 2.98x and 2.24x speedup in the prompt processing and text generation tasks in llama.cpp, respectively, and an improvement of 6% in StableDiffusion.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Design and Implementation of a High-Performance Operator Library for RISC-V Vector Extension

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Wenzhu Wang,
  • Ke Li,
  • Qingbo Wu,
  • Jie Yu,
  • Xiaodong Liu,
  • Lili Liu,
  • Zhilong Liu,
  • Jianfeng Li,
  • Min Liu

摘要

The RISC-V instruction set architecture is gaining increasing attention in embedded and edge computing due to its open source, AI scalability, and low power consumption, which make it well-suited for edge-side AI applications. However, the standard operator library is not optimized for RISC-V well, which results in suboptimal performance due to the underutilization of the AI scalability. Therefore, using the RISC-V Vector (RVV) extension, we design and implement a high-performance operator library to address this challenge and enhance operator computing efficiency. The library is compatible with most inference frameworks and different RVV versions. Meanwhile, it accelerates several operations, such as convolution, activation functions, and basic arithmetic, by converting scalar calculations into vector calculations through loop expansion. We compared the performance of our operator library across three types of applications: YOLOv8n, llama.cpp, and StableDiffusion. The results show that, compared to scalar implementation, our approach achieves a performance improvement of 32.09% to 63.97% in YOLOv8n, a 2.98x and 2.24x speedup in the prompt processing and text generation tasks in llama.cpp, respectively, and an improvement of 6% in StableDiffusion.