Global Static Path Planning for Unmanned Surface Vehicle Based on Hierarchical Deep Reinforcement Learning
摘要
The Unmanned Surface Vehicle (USV) has garnered widespread attention across numerous sectors, such as military operations and scientific research endeavors. A dependable autonomous path planning system is an indispensable requirement for ensuring the safe navigation of USVs. Deep reinforcement learning algorithms, representing a pivotal category within the sphere of artificial intelligence, have drawn extensive and focused interest due to their fusion of reinforcement learning methodologies with deep learning approaches, thus showcasing remarkable sensing and decision-making faculties. In this study, we adopt an enhanced hierarchical deep reinforcement learning variant of the DQN algorithm. The strategy involves initially extracting a relatively advantageous path from a less intricate maritime scenario and attributing additional reward values to each point along this path, serving as a foundational body of prior knowledge for subsequent training phases on more sophisticated and fine-grained nautical charts. By strategically leveraging these incentivized rewards, our method efficiently generates a reliable navigation path within the context of high-detail maritime charts. This innovative solution effectively addresses the challenge of complex training under expansive nautical map conditions, ultimately achieving safe and economically viable global static path planning from origin to destination.