Scholarlink

<p>Autonomous driving technology faces significant safety challenges, particularly at unsignalized intersections. Centralized cooperative methods have been developed to manage the flow of connected and automated vehicles. However, many existing approaches depend on basic control algorithms, leading to lengthy inference time and suboptimal solutions. Consequently, real-time performance, road resource utilization, and traffic efficiency are compromised. While some studies have integrated reinforcement learning (RL) techniques to address these issues, they often comprise safety due to reward-driven optimization and the oversimplification of traffic scenarios, such as designing specific flow directions. These limitations raise concerns about their real-world applicability and safety. To address these shortcomings, this paper introduces a novel behavior-constrained proximal policy optimization (BCPPO) method for RL-based cooperative vehicle control at intersections. First, the problem is formulated as a multi-agent RL task within a Markov Game (MG) framework. A multi-agent proximal policy optimization (MAPPO) algorithm is proposed to handle the complex cooperative dynamics among multiple agents. The policy network employs a Long Short-Term Memory (LSTM) encoder to capture extensive social interaction information among the agents. Second, intersection control problem is formalized within the MG framework, and a safety-enhanced cooperative vehicle control strategy, BCPPO, is proposed. This method integrates formal safety verification and behavior constraints into the training and deployment of MAPPO to ensure safety and robustness. Finally, extensive simulation experiments are conducted across various intersection scenarios to evaluate the performance of BCPPO against RL-based proximal policy optimization (PPO), the rule-based first-come-first-served (FCFS) method, and the optimal control (OC)-based vehicles-intersection control system (VICS). The results demonstrate that BCPPO achieves a zero-collision rate during deployment and enhances driving comfort by 60.75%, compared to the non-safety-aware PPO method, which has a collision rate of about 13.85%. Furthermore, BCPPO improves traffic efficiency by 16.15% in comparison to FCFS and reduces inference time by a factor of 71.73 relative to the VICS method.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Behavior-Constrained Multi-Agent Proximal Policy Optimization for Cooperative Vehicle Control at Future Intersections

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年10月1日

查看全文
  • Zhenhai Gao,
  • Hesheng Hao,
  • Fei Gao,
  • Rui Zhao

摘要

Autonomous driving technology faces significant safety challenges, particularly at unsignalized intersections. Centralized cooperative methods have been developed to manage the flow of connected and automated vehicles. However, many existing approaches depend on basic control algorithms, leading to lengthy inference time and suboptimal solutions. Consequently, real-time performance, road resource utilization, and traffic efficiency are compromised. While some studies have integrated reinforcement learning (RL) techniques to address these issues, they often comprise safety due to reward-driven optimization and the oversimplification of traffic scenarios, such as designing specific flow directions. These limitations raise concerns about their real-world applicability and safety. To address these shortcomings, this paper introduces a novel behavior-constrained proximal policy optimization (BCPPO) method for RL-based cooperative vehicle control at intersections. First, the problem is formulated as a multi-agent RL task within a Markov Game (MG) framework. A multi-agent proximal policy optimization (MAPPO) algorithm is proposed to handle the complex cooperative dynamics among multiple agents. The policy network employs a Long Short-Term Memory (LSTM) encoder to capture extensive social interaction information among the agents. Second, intersection control problem is formalized within the MG framework, and a safety-enhanced cooperative vehicle control strategy, BCPPO, is proposed. This method integrates formal safety verification and behavior constraints into the training and deployment of MAPPO to ensure safety and robustness. Finally, extensive simulation experiments are conducted across various intersection scenarios to evaluate the performance of BCPPO against RL-based proximal policy optimization (PPO), the rule-based first-come-first-served (FCFS) method, and the optimal control (OC)-based vehicles-intersection control system (VICS). The results demonstrate that BCPPO achieves a zero-collision rate during deployment and enhances driving comfort by 60.75%, compared to the non-safety-aware PPO method, which has a collision rate of about 13.85%. Furthermore, BCPPO improves traffic efficiency by 16.15% in comparison to FCFS and reduces inference time by a factor of 71.73 relative to the VICS method.