Scholarlink

<p>Learning effective policies under sparse reward conditions remains one of the core challenges in deep reinforcement learning (DRL). Particularly in unmanned aerial vehicle (UAV) pursuit-evasion games, the high dynamics and uncertainty inherent to the environment are further exacerbated by sparse rewards, imposing strict requirements on exploration efficiency, training efficiency, and real-time response capabilities. To address these challenges, we propose a novel DRL approach. Specifically, to overcome the exploration dilemma induced by sparse rewards, causal reasoning is introduced to develop an intrinsic reward function based on causal influence (CI). By leveraging causal interventions and conditional mutual information, we reliably estimate and calculate the CI of the UAV’s actions on the next states, using it to guide effective exploration. To further address the training inefficiency caused by excessive negative reward samples in sparse reward environments, we design an attraction-based prioritized experience replay (ABPER) mechanism. This mechanism constructs a trajectory attraction function that quantifies the attraction between the pursuer and the evader within a virtual potential field. Trajectory priorities are calculated to prioritize replay of high-value experiences, enhancing sample efficiency and further improving the UAV’s decision-making capability and task success rate. We integrate the deep deterministic policy gradient (DDPG) algorithm with the CI-based intrinsic reward and the ABPER mechanism to develop a new algorithm, named DDPG-CI-ABPER. Experimental results demonstrate that our method significantly outperforms baselines and state-of-the-art (SOTA) RL algorithms in both UAV navigation with obstacle avoidance and pursuit-evasion scenarios. It achieves a success rate of 98.5% in navigation environments. In pursuit-evasion scenarios where the pursuer and evader possess comparable performance, it is 92.9%, proving its superior algorithmic performance and real-time decision-making capability.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Causal reinforcement learning for unmanned aerial vehicle pursuit-evasion games with sparse rewards

  • 期刊论文

  • 发表日期：

    2026年1月24日

查看全文
  • Lei Lei,
  • Zhaoxin Dong,
  • Chengfu Wu,
  • Huaimin Chen

摘要

Learning effective policies under sparse reward conditions remains one of the core challenges in deep reinforcement learning (DRL). Particularly in unmanned aerial vehicle (UAV) pursuit-evasion games, the high dynamics and uncertainty inherent to the environment are further exacerbated by sparse rewards, imposing strict requirements on exploration efficiency, training efficiency, and real-time response capabilities. To address these challenges, we propose a novel DRL approach. Specifically, to overcome the exploration dilemma induced by sparse rewards, causal reasoning is introduced to develop an intrinsic reward function based on causal influence (CI). By leveraging causal interventions and conditional mutual information, we reliably estimate and calculate the CI of the UAV’s actions on the next states, using it to guide effective exploration. To further address the training inefficiency caused by excessive negative reward samples in sparse reward environments, we design an attraction-based prioritized experience replay (ABPER) mechanism. This mechanism constructs a trajectory attraction function that quantifies the attraction between the pursuer and the evader within a virtual potential field. Trajectory priorities are calculated to prioritize replay of high-value experiences, enhancing sample efficiency and further improving the UAV’s decision-making capability and task success rate. We integrate the deep deterministic policy gradient (DDPG) algorithm with the CI-based intrinsic reward and the ABPER mechanism to develop a new algorithm, named DDPG-CI-ABPER. Experimental results demonstrate that our method significantly outperforms baselines and state-of-the-art (SOTA) RL algorithms in both UAV navigation with obstacle avoidance and pursuit-evasion scenarios. It achieves a success rate of 98.5% in navigation environments. In pursuit-evasion scenarios where the pursuer and evader possess comparable performance, it is 92.9%, proving its superior algorithmic performance and real-time decision-making capability.