Scholarlink

<p>Recent developments indicate a shift toward web services that employ ever-larger AI models, e.g., large language models (LLMs), requiring powerful and expensive hardware for inference. High-performance computing (HPC) systems are commonly equipped with such hardware for the purpose of large-scale computation tasks. However, HPC infrastructure is inherently unsuitable for hosting real-time web services due to network, security, and scheduling constraints. While various efforts exist to integrate external scheduling solutions, these often require compromises in terms of security or usability for existing HPC users. In this paper, we present SAIA, a Slurm-native platform consisting of a scheduler and a proxy. The scheduler interacts with Slurm to ensure the availability and scalability of services, while the proxy provides external access, which is secured via confined SSH commands. We have demonstrated SAIA’s applicability by deploying a large-scale LLM web service that has served over 100&#xa0;000 users.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

SAIA: a seamless Slurm-native solution for HPC-based services

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年5月8日

查看全文
  • Ali Doosthosseini,
  • Jonathan Decker,
  • Hendrik Nolte,
  • Julian Kunkel

摘要

Recent developments indicate a shift toward web services that employ ever-larger AI models, e.g., large language models (LLMs), requiring powerful and expensive hardware for inference. High-performance computing (HPC) systems are commonly equipped with such hardware for the purpose of large-scale computation tasks. However, HPC infrastructure is inherently unsuitable for hosting real-time web services due to network, security, and scheduling constraints. While various efforts exist to integrate external scheduling solutions, these often require compromises in terms of security or usability for existing HPC users. In this paper, we present SAIA, a Slurm-native platform consisting of a scheduler and a proxy. The scheduler interacts with Slurm to ensure the availability and scalability of services, while the proxy provides external access, which is secured via confined SSH commands. We have demonstrated SAIA’s applicability by deploying a large-scale LLM web service that has served over 100 000 users.