DNN-Schedule: A Predictive Scheduler for Minimizing Interference of Co-located DNN Workload
摘要
The GPU resource scheduler constitutes a crucial component of the cluster system, tasked with the management and scheduling of diverse deep learning workloads. Existing schedulers exhibit certain limitations, such as intrusive methods, limited scalability, and low generalization. These lead to extra operational costs and low utilization for GPU clusters, as well as hindering the efficiency of deep neural networks (DNNs) development and application. In this paper, we present DNN-schedule, a scheduler aimed at minimizing interference of co-located deep learning jobs under multiple predictive performance metrics. We adopt a predictive model based on graph neural networks to forecast GPU resource utilization and task runtime. We then devise a priority-based job co-location method, which formulates decisions from attributes of the jobs and their predicted performance, therefore achieving the traits of non-intrusiveness and interference-awareness. Extensive experimental simulations demonstrate that DNN-schedule outperforms existing deep learning resource schedulers in terms of average job completion time, queuing time, and P99 tail latency.