Scholarlink

We propose a latency-optimized neural network model to dynamically predict job execution times for the ALICE experiment at CERN, replacing static Time-To-Live (TTL) allocations. Utilizing Nvidia A100 GPUs, we optimize inference latency via FP16 and INT8 quantization, 2:4 sparsity, quantization-aware training, and graph compilation. Results show that FP16 and sparsity reduce latency for larger batches, while INT8 is optimal for single-sample predictions. For single-sample online inference, static INT8 quantization achieves a median 0.38 ms prediction time, a 1.8x improvement over the 0.71 ms baseline. The model achieves a 1.9-hour RMSE, improving on the 14.23-hour RMSE of current TTL assignments. With sub-40ms inference latency on GPU hardware, this work demonstrates how NN optimization can help achieve performance demands of large-scale distributed computing systems.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Fast Prediction of Job Execution Times in the ALICE Grid Through GPU-Based Inference with Quantization and Sparsity Techniques

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Tomasz Lelek,
  • Szymon Mazurek,
  • Maciej Wielgosz,
  • Bartosz Balis

摘要

We propose a latency-optimized neural network model to dynamically predict job execution times for the ALICE experiment at CERN, replacing static Time-To-Live (TTL) allocations. Utilizing Nvidia A100 GPUs, we optimize inference latency via FP16 and INT8 quantization, 2:4 sparsity, quantization-aware training, and graph compilation. Results show that FP16 and sparsity reduce latency for larger batches, while INT8 is optimal for single-sample predictions. For single-sample online inference, static INT8 quantization achieves a median 0.38 ms prediction time, a 1.8x improvement over the 0.71 ms baseline. The model achieves a 1.9-hour RMSE, improving on the 14.23-hour RMSE of current TTL assignments. With sub-40ms inference latency on GPU hardware, this work demonstrates how NN optimization can help achieve performance demands of large-scale distributed computing systems.