Scientists expect HPC systems to support scalable jobs, deliver consistent time-to-solution, and offer flexibility in software and hardware configurations. Accommodating this combination of scalable jobs, reproducible performance, and custom environments is particularly challenging for HPC service providers operating large-scale, general-purpose supercomputers. The Alps infrastructure introduces a novel approach to address these challenges by using versatile software defined cluster (vCluster) technology to create isolated, customizable environments tailored to specific research communities. However, when defined as static resource sets, such partitions limit the ability to accommodate highly scalable jobs or respond dynamically to workload changes. To address this, we propose a solution that enables elastic resource sharing across scientific platforms within the management stack of the supercomputing system. By replaying job traces from a production supercomputer, we demonstrate that our approach can meet multiple quality-of-service (QoS) objectives while maintaining high overall resource utilization.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Resource Elasticity for Scientific Platforms on HPC Infrastructure

  • Maxime Martinasso,
  • Thomas C. Schulthess

摘要

Scientists expect HPC systems to support scalable jobs, deliver consistent time-to-solution, and offer flexibility in software and hardware configurations. Accommodating this combination of scalable jobs, reproducible performance, and custom environments is particularly challenging for HPC service providers operating large-scale, general-purpose supercomputers. The Alps infrastructure introduces a novel approach to address these challenges by using versatile software defined cluster (vCluster) technology to create isolated, customizable environments tailored to specific research communities. However, when defined as static resource sets, such partitions limit the ability to accommodate highly scalable jobs or respond dynamically to workload changes. To address this, we propose a solution that enables elastic resource sharing across scientific platforms within the management stack of the supercomputing system. By replaying job traces from a production supercomputer, we demonstrate that our approach can meet multiple quality-of-service (QoS) objectives while maintaining high overall resource utilization.