Scholarlink

Large Language Models (LLMs) built on Transformer architectures have achieved remarkable success in Natural Language Processing (NLP) applications. However, as the scale of model parameters continues to expand, deployment becomes increasingly challenging, particularly for resource-constrained edge devices. Emerging dense architectures such as Monarch Mixer, offer a promising alternative by achieving superior accuracy with fewer parameters and reduced computational demands compared to conventional transformers. Nevertheless, Monarch Mixer’s memory-intensive operators result in inefficient hardware utilization and increased latency, especially in small-scale text inference scenarios. To address this limitation, we propose FPAMM, a fine-grained pipeline accelerator that exploits Monarch Mixer’s operator dependencies through a fine-grained pipeline architecture. Our approach incorporates key optimizations such as operator fusion and dual buffering to enhance data flow efficiency. Concurrently, we optimized resource allocation and ensured load balancing through design space exploration. Deployed on Xilinx Alveo U280 platform, FPAMM achieves 2.21–40.42 \(\times \) speedup for a 12-layer M2-BERT compared to CPU and GPU baselines. When evaluated against existing FPGA-based Transformer accelerators, FPAMM delivers superior performance, with 7.5 \(\times \) , 8.35 \(\times \) and 2.28 \(\times \) improvements in throughput compared to UPSA, TRAC and FQ-BERT respectively, while maintaining the lowest latency. Through innovative fine-grained pipelining and operator fusion techniques, FPAMM mitigates the memory overhead inherent in Monarch Mixer. Experimental results confirm FPAMM’s potential as a promising solution for low-latency, high-throughput edge deployment.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

FPAMM: Fine-Grained Pipeline Architecture Accelerator for the Novel Transformer Architecture - Monarch Mixer

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Hanyuan Li,
  • Mingche Lai,
  • Xingyun Qi,
  • Puguang Liu,
  • Qiang Wang,
  • Zhenqi Li,
  • Yihang Lu,
  • Yuan Li

摘要

Large Language Models (LLMs) built on Transformer architectures have achieved remarkable success in Natural Language Processing (NLP) applications. However, as the scale of model parameters continues to expand, deployment becomes increasingly challenging, particularly for resource-constrained edge devices. Emerging dense architectures such as Monarch Mixer, offer a promising alternative by achieving superior accuracy with fewer parameters and reduced computational demands compared to conventional transformers. Nevertheless, Monarch Mixer’s memory-intensive operators result in inefficient hardware utilization and increased latency, especially in small-scale text inference scenarios. To address this limitation, we propose FPAMM, a fine-grained pipeline accelerator that exploits Monarch Mixer’s operator dependencies through a fine-grained pipeline architecture. Our approach incorporates key optimizations such as operator fusion and dual buffering to enhance data flow efficiency. Concurrently, we optimized resource allocation and ensured load balancing through design space exploration. Deployed on Xilinx Alveo U280 platform, FPAMM achieves 2.21–40.42 \(\times \) speedup for a 12-layer M2-BERT compared to CPU and GPU baselines. When evaluated against existing FPGA-based Transformer accelerators, FPAMM delivers superior performance, with 7.5 \(\times \) , 8.35 \(\times \) and 2.28 \(\times \) improvements in throughput compared to UPSA, TRAC and FQ-BERT respectively, while maintaining the lowest latency. Through innovative fine-grained pipelining and operator fusion techniques, FPAMM mitigates the memory overhead inherent in Monarch Mixer. Experimental results confirm FPAMM’s potential as a promising solution for low-latency, high-throughput edge deployment.