The Unmanned Surface Vehicle (USV) has garnered widespread attention across numerous sectors, such as military operations and scientific research endeavors. A dependable autonomous path planning system is an indispensable requirement for ensuring the safe navigation of USVs. Deep reinforcement learning algorithms, representing a pivotal category within the sphere of artificial intelligence, have drawn extensive and focused interest due to their fusion of reinforcement learning methodologies with deep learning approaches, thus showcasing remarkable sensing and decision-making faculties. In this study, we adopt an enhanced hierarchical deep reinforcement learning variant of the DQN algorithm. The strategy involves initially extracting a relatively advantageous path from a less intricate maritime scenario and attributing additional reward values to each point along this path, serving as a foundational body of prior knowledge for subsequent training phases on more sophisticated and fine-grained nautical charts. By strategically leveraging these incentivized rewards, our method efficiently generates a reliable navigation path within the context of high-detail maritime charts. This innovative solution effectively addresses the challenge of complex training under expansive nautical map conditions, ultimately achieving safe and economically viable global static path planning from origin to destination.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Global Static Path Planning for Unmanned Surface Vehicle Based on Hierarchical Deep Reinforcement Learning

  • Zhiyang Bao,
  • Yunsheng Fan,
  • Zhe Sun

摘要

The Unmanned Surface Vehicle (USV) has garnered widespread attention across numerous sectors, such as military operations and scientific research endeavors. A dependable autonomous path planning system is an indispensable requirement for ensuring the safe navigation of USVs. Deep reinforcement learning algorithms, representing a pivotal category within the sphere of artificial intelligence, have drawn extensive and focused interest due to their fusion of reinforcement learning methodologies with deep learning approaches, thus showcasing remarkable sensing and decision-making faculties. In this study, we adopt an enhanced hierarchical deep reinforcement learning variant of the DQN algorithm. The strategy involves initially extracting a relatively advantageous path from a less intricate maritime scenario and attributing additional reward values to each point along this path, serving as a foundational body of prior knowledge for subsequent training phases on more sophisticated and fine-grained nautical charts. By strategically leveraging these incentivized rewards, our method efficiently generates a reliable navigation path within the context of high-detail maritime charts. This innovative solution effectively addresses the challenge of complex training under expansive nautical map conditions, ultimately achieving safe and economically viable global static path planning from origin to destination.