Scholarlink

This paper suggests the formulation of a simulation system to test auto-scaling in cloud inference AI settings. The dynamic workload conditions were modelled by a synthetic request generation process, and realistic GPU cluster trace dataset. The following three scaling policies were conducted and contrasted: (i) a rule-based controller of a reactive nature, (ii) a predictive workload forecast-based policy, and (iii) an RL-based policy aimed to minimize latency and utilization. The simulation model was set in a manner that resembled real-time scaling and its implication on service-level agreement (SLA) compliance, response time, and infrastructure expenditure. Extended experiments have been carried out on both synthetic and real data to analyze the robustness and trade-offs of every policy under different load characteristics. The test outcomes show that it is possible to simulate the cloud-native type of scaling with a low-cost, repeatable environment without using the actual infrastructure. The constructed framework provides a usable and extendable device to analyze scaling logic into AI-serving pipelines.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A Simulation Framework for Evaluating Auto-scaling Strategies in Cloud-Based AI Inference Workloads

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Pavan Nutalapati,
  • Lahari Putty,
  • Arun Kumar Elengovan,
  • Nandagopal Seshagiri

摘要

This paper suggests the formulation of a simulation system to test auto-scaling in cloud inference AI settings. The dynamic workload conditions were modelled by a synthetic request generation process, and realistic GPU cluster trace dataset. The following three scaling policies were conducted and contrasted: (i) a rule-based controller of a reactive nature, (ii) a predictive workload forecast-based policy, and (iii) an RL-based policy aimed to minimize latency and utilization. The simulation model was set in a manner that resembled real-time scaling and its implication on service-level agreement (SLA) compliance, response time, and infrastructure expenditure. Extended experiments have been carried out on both synthetic and real data to analyze the robustness and trade-offs of every policy under different load characteristics. The test outcomes show that it is possible to simulate the cloud-native type of scaling with a low-cost, repeatable environment without using the actual infrastructure. The constructed framework provides a usable and extendable device to analyze scaling logic into AI-serving pipelines.