Such a chapter proposes a simulation framework to the dynamic hybrid edge-cloud offloading of AI inference workloads to tackle the issue of latency and the optimization of resources. Inference is performed locally on a CPU on a specific device or remotely on a GPU, depending on the estimated latency, batch size and network conditions in real time. In static and dynamic offloading conditions, ResNet18 architecture was applied on the CIFAR-10 dataset. To simulate the general deployment constraints of the real world, artificial thresholds of latency, network charges, and synthetics were added to workload variations. The analysis of experiments established the feasibility of the adaptive offloading with conditions identified under which edge, cloud, or balanced decisions arose. The suggested structure allows simulating intelligent and cost-efficient policies of intelligent AI offloading without any physical edge or cloud infrastructure.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Adaptive Offloading Strategies for Hybrid Edge-Cloud AI Inference: A Latency-Aware Simulation Framework

  • Lahari Putty,
  • Pavan Nutalapati,
  • Rakesh Keshava,
  • Arun Kumar Elengovan

摘要

Such a chapter proposes a simulation framework to the dynamic hybrid edge-cloud offloading of AI inference workloads to tackle the issue of latency and the optimization of resources. Inference is performed locally on a CPU on a specific device or remotely on a GPU, depending on the estimated latency, batch size and network conditions in real time. In static and dynamic offloading conditions, ResNet18 architecture was applied on the CIFAR-10 dataset. To simulate the general deployment constraints of the real world, artificial thresholds of latency, network charges, and synthetics were added to workload variations. The analysis of experiments established the feasibility of the adaptive offloading with conditions identified under which edge, cloud, or balanced decisions arose. The suggested structure allows simulating intelligent and cost-efficient policies of intelligent AI offloading without any physical edge or cloud infrastructure.