To address the challenge of autonomous learning and evolution of strategies for both pursuer and evader spacecraft, a self-iterative pursuit-evasion strategy based on population reinforcement learning is proposed. Considering the constraints and stochastic factors of the pursuer and evader, a model of typical spacecraft pursuit-evasion process is constructed. On this foundation, a population-based reinforcement learning method is developed to iteratively update the strategies of both the pursuer and evader simultaneously. Each generation of strategy learning is implemented using the double truncated proximal policy optimization, while the next generation of strategy iteration adopts a weighted self-play approach for opponent selection. An orbital-level pursuit-evasion simulation environment and a distributed training framework for reinforcement learning are used to support learning and iteration of strategies for both the pursuer and evader. Finally, simulation experiments are conducted, which comprehensively validate the effectiveness and superiority of the proposed method.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Population Reinforcement Learning Method for Pursuit-Evasion Game of the Spacecraft

  • Jiahong Jiang,
  • Hua Zheng,
  • Togntong Yu,
  • Qiyue Yin

摘要

To address the challenge of autonomous learning and evolution of strategies for both pursuer and evader spacecraft, a self-iterative pursuit-evasion strategy based on population reinforcement learning is proposed. Considering the constraints and stochastic factors of the pursuer and evader, a model of typical spacecraft pursuit-evasion process is constructed. On this foundation, a population-based reinforcement learning method is developed to iteratively update the strategies of both the pursuer and evader simultaneously. Each generation of strategy learning is implemented using the double truncated proximal policy optimization, while the next generation of strategy iteration adopts a weighted self-play approach for opponent selection. An orbital-level pursuit-evasion simulation environment and a distributed training framework for reinforcement learning are used to support learning and iteration of strategies for both the pursuer and evader. Finally, simulation experiments are conducted, which comprehensively validate the effectiveness and superiority of the proposed method.