Most current job schedulers determine the job schedule and resources based on first-come-first-served with backfilling policy. However, the majority of the current job schedulers do not consider the communication pattern of a job. This may result in performance slowdown due to job interference. We propose and implement Combal (Communication Balanced) algorithm in the SLURM job scheduler. Combal allocates resources based on the communication profile of a job and the current cluster state. Combal aims to reduce the communication cost by allocating frequently communicating processes on compute nodes of the same leaf switch of a fat-tree. We compare this with TopoMatch algorithm using real job logs from the Intrepid, Mira, and Theta supercomputers. We observed up to 20% reduction in queue waiting times due to faster runtimes owing to lower communication times of jobs.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Communication-Balanced Job Allocation Using SLURM

  • Gagandeep Mangat,
  • Preeti Malakar

摘要

Most current job schedulers determine the job schedule and resources based on first-come-first-served with backfilling policy. However, the majority of the current job schedulers do not consider the communication pattern of a job. This may result in performance slowdown due to job interference. We propose and implement Combal (Communication Balanced) algorithm in the SLURM job scheduler. Combal allocates resources based on the communication profile of a job and the current cluster state. Combal aims to reduce the communication cost by allocating frequently communicating processes on compute nodes of the same leaf switch of a fat-tree. We compare this with TopoMatch algorithm using real job logs from the Intrepid, Mira, and Theta supercomputers. We observed up to 20% reduction in queue waiting times due to faster runtimes owing to lower communication times of jobs.