Scholarlink

This paper presents an optimized VLSI architecture for implementing transformer-based neural networks with emphasis on hardware efficiency and reduced computational complexity. The proposed design features a six layered architecture with a novel 512-dimensional processing unit utilizing an innovative matrix computation approach that achieves significant power savings. Our implementation leverages COordinate Rotation DIgital Computer (CORDIC) architecture to efficiently compute the trigonometric functions required for positional encoding in the transformer model. The design incorporates two specialized CORDIC modules: one configured in rotation mode for sine/cosine calculations, and another in vectoring mode for division operations, both crucial for positional encoding and softmax computations respectively. The architecture is implemented on Vitrex Ultrascale+ VU37P FPGA which demonstrates improved resource utilization through specialized matrix multiplication units while maintaining the model’s core functionality for natural language processing tasks. The implementation achieves a 30% reduction in power consumption and LUT utilization compared to conventional implementations reduced over 48%.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Hardware-Efficient VLSI Implementation of Self-attention Transformers for Natural Language Processing

  • 章节

  • 发表日期：

    2026年

查看全文
  • Mogal Riyaz,
  • P. Naveen Kumar,
  • I. Saikrishna,
  • R. Saiteja

摘要

This paper presents an optimized VLSI architecture for implementing transformer-based neural networks with emphasis on hardware efficiency and reduced computational complexity. The proposed design features a six layered architecture with a novel 512-dimensional processing unit utilizing an innovative matrix computation approach that achieves significant power savings. Our implementation leverages COordinate Rotation DIgital Computer (CORDIC) architecture to efficiently compute the trigonometric functions required for positional encoding in the transformer model. The design incorporates two specialized CORDIC modules: one configured in rotation mode for sine/cosine calculations, and another in vectoring mode for division operations, both crucial for positional encoding and softmax computations respectively. The architecture is implemented on Vitrex Ultrascale+ VU37P FPGA which demonstrates improved resource utilization through specialized matrix multiplication units while maintaining the model’s core functionality for natural language processing tasks. The implementation achieves a 30% reduction in power consumption and LUT utilization compared to conventional implementations reduced over 48%.