Collaborative Inference for Adaptive DNN Pipeline-Aware Alignment
摘要
Recent advancements in Internet of Things (IoT) and deep neural network (DNN) have enabled spatio-temporal data-based inference tasks with substantial computational demands. Edge computing mitigates these demands by offloading DNN tasks to edge servers. However, existing studies mainly focus on single-task optimization for delay reduction, neglecting system throughput and energy efficiency, which is critical for device lifespan and performance. Our work addresses the high throughput and real-time requirements of DNN inference tasks for spatio-temporal data, while reducing system energy overhead and ensuring task accuracy. We employ early exit techniques and consider inter-task parallelism to optimize energy consumption and enhance system throughput. We refine the original flow graph and achieve efficient pipeline alignment through effective scheduling. When selecting early exit models, we only consider tasks that meet accuracy requirements, thus avoiding models with poor inference accuracy. Experimental results show that compared with the existing state-of-the-art schemes, our method improves the throughput by 14.2% to 60% on models such as VGG19, Googlenet and Resnet-50, and reduces the average inference energy consumption of tasks by about 11.1% to 49.8%.