Computation offloading in vehicular communications using PPO-based deep reinforcement learning
摘要
Vehicular edge computing enables vehicles to offload compute-intensive tasks to nearby edge servers integrated with roadside units to enhance performance and conserve battery power. In practical environments, vehicles often experience intrinsic task dependencies, stringent deadlines, and speeds and locations within highly dynamic networks, which is still a significant challenge in vehicular communications. Numerical optimization, game theory, and heuristic schemes struggle to meet these diverse and dynamic requirements in optimizing the computation offloading decisions. To this end, we propose a Proximal Policy Optimization (PPO)-based algorithm that optimally manages offloading decisions by employing a policy gradient approach with surrogate clipping to ensure stable and reliable updates. This is crucial in dynamic vehicular networks, where policy updates can significantly impact system performance. Furthermore, we use Generalized Advantage Estimation to further enhance the stability and efficiency by accurately estimating advantages over multiple steps. Our proposed PPO algorithm effectively balances exploration and exploitation and yields optimal offloading decisions while minimizing delays and dropped task ratio. Extensive experiments validate our approach’s efficacy, demonstrating substantial improvements in task completion rates and minimizing delays while meeting the requirements of intrinsic task dependencies and stringent deadlines in dynamic vehicular setups. For example, results show that the proposed method surpasses DQN by