Scholarlink

<p>The growing demand for large-scale artificial intelligence (AI) and deep learning (DL) has boosted rapid development of increasingly large AI clusters. Benchmarking such systems is non-trivial and critical for enabling fair comparison, revealing system bottlenecks, and guiding optimization. However, existing benchmarks suffer from key limitations. Traditional high-performance computing (HPC) benchmarks, such as the de facto standard LINPACK, lack representative AI workloads. Popular component AI benchmarks, such as MLPerf and AIBench, adopt representative DL tasks but still suffer from limited scalability and affordability. AIPerf addresses these limitations but fails to reveal system bottlenecks and represent real-world distributed training dynamics in AI clusters with certain network conditions. To address these issues, we propose ScaleBench, a scalable and affordable benchmark for evaluating AI clusters. We use foundational DL operators as the representative workloads. We design scalable workflows with data parallelism and model parallelism that automatically adapt to clusters of various scales and support affordable execution. We define simple metrics of performance and energy efficiency, enabling easy comparison between different systems. We evaluate ScaleBench on various GPU configurations and communication conditions to ensure its scalability, affordability, and stability, as well as its capability to reveal system bottlenecks. The results demonstrate that ScaleBench offers a practical and insightful tool for benchmarking and evaluating modern AI clusters.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

ScaleBench: a scalable and affordable benchmark for evaluating AI clusters

  • 期刊论文

  • 发表日期：

    2026年7月24日

查看全文
  • Boqi Chen,
  • Huikang Huang,
  • Haolang Yin,
  • Xiaoxuan Luo,
  • Weiwei Lin,
  • Qing Ding

摘要

The growing demand for large-scale artificial intelligence (AI) and deep learning (DL) has boosted rapid development of increasingly large AI clusters. Benchmarking such systems is non-trivial and critical for enabling fair comparison, revealing system bottlenecks, and guiding optimization. However, existing benchmarks suffer from key limitations. Traditional high-performance computing (HPC) benchmarks, such as the de facto standard LINPACK, lack representative AI workloads. Popular component AI benchmarks, such as MLPerf and AIBench, adopt representative DL tasks but still suffer from limited scalability and affordability. AIPerf addresses these limitations but fails to reveal system bottlenecks and represent real-world distributed training dynamics in AI clusters with certain network conditions. To address these issues, we propose ScaleBench, a scalable and affordable benchmark for evaluating AI clusters. We use foundational DL operators as the representative workloads. We design scalable workflows with data parallelism and model parallelism that automatically adapt to clusters of various scales and support affordable execution. We define simple metrics of performance and energy efficiency, enabling easy comparison between different systems. We evaluate ScaleBench on various GPU configurations and communication conditions to ensure its scalability, affordability, and stability, as well as its capability to reveal system bottlenecks. The results demonstrate that ScaleBench offers a practical and insightful tool for benchmarking and evaluating modern AI clusters.