Scholarlink

<p>Vision Transformer (ViT) is the key to state-of-the-art DNN models in computer vision. Due to the high memory requirement and computational burden of ViTs, a lightweight design and integer quantization are essential approaches . However, these lightweight implementations still target desktop-class hardware and high-end edge devices. The deployment on resource-restricted embedded devices remains a significant challenge. In this context, floating-point arithmetic avoidance, buffer management, and CPU’s SIMD leverage cannot be ignored to meet embedded system requirements, such as memory footprint and real-time execution. Therefore, we implement an ultra-lightweight ViT, i.e., a fully integer CNN-Transformer hybrid model with Simple attention by Post-Training Quantization (PTQ), targeting embedded hardware. Specifically, we introduce Normalization Equivalent Quantization (NEQ) to reduce <i>l</i>-1 normalization computation, and buffer optimization, e.g., Attention row buffer to mitigate <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(O(n^2)\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi>O</mi> <mo stretchy="false">(</mo> <msup> <mi>n</mi> <mn>2</mn> </msup> <mo stretchy="false">)</mo> </mrow> </math></EquationSource> </InlineEquation> buffer size increase. As a result, the parameter size can be kept under 512 KiB and can be deployed in on-chip memory on a small-scale RISC-V core, CV32E40p. Additionally, with SIMD extension and its dedicated kernels, our implementation achieves 10.4<InlineEquation ID="IEq2"> <EquationSource Format="TEX">\(\times \)</EquationSource> <EquationSource Format="MATHML"><math> <mo>×</mo> </math></EquationSource> </InlineEquation> acceleration for the logit of Simple attention, resulting in 1.4<InlineEquation ID="IEq3"> <EquationSource Format="TEX">\(\times \)</EquationSource> <EquationSource Format="MATHML"><math> <mo>×</mo> </math></EquationSource> </InlineEquation> latency improvement compared with the combination of typical quantization and the cutting-edge Integer ViT module.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

An embedded vision transformer with quantization-friendly simple attention, optimized buffers, and SIMD acceleration

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年4月9日

查看全文
  • Hayata Kaneko,
  • Lin Meng

摘要

Vision Transformer (ViT) is the key to state-of-the-art DNN models in computer vision. Due to the high memory requirement and computational burden of ViTs, a lightweight design and integer quantization are essential approaches . However, these lightweight implementations still target desktop-class hardware and high-end edge devices. The deployment on resource-restricted embedded devices remains a significant challenge. In this context, floating-point arithmetic avoidance, buffer management, and CPU’s SIMD leverage cannot be ignored to meet embedded system requirements, such as memory footprint and real-time execution. Therefore, we implement an ultra-lightweight ViT, i.e., a fully integer CNN-Transformer hybrid model with Simple attention by Post-Training Quantization (PTQ), targeting embedded hardware. Specifically, we introduce Normalization Equivalent Quantization (NEQ) to reduce l-1 normalization computation, and buffer optimization, e.g., Attention row buffer to mitigate \(O(n^2)\) O ( n 2 ) buffer size increase. As a result, the parameter size can be kept under 512 KiB and can be deployed in on-chip memory on a small-scale RISC-V core, CV32E40p. Additionally, with SIMD extension and its dedicated kernels, our implementation achieves 10.4 \(\times \) × acceleration for the logit of Simple attention, resulting in 1.4 \(\times \) × latency improvement compared with the combination of typical quantization and the cutting-edge Integer ViT module.