<p>Path planning is the essential requirement for autonomous and collision-free movement of a robot agent in the environment. In this paper, the deep Q-network (DQN) and its variants double deep Q-network (DDQN) and duelling double deep Q-network (D3QN) algorithms are considered with a modified reward function for efficient navigation and goal searching system. In the proposed work, initially a base reward function is considered and subsequently an updated reward function is designed to compare the learning’s of an agent’s actions like moving closer to the goal or changing its orientation in reaching the goal to systematically analyze the behavior of the agent. The base reward function consists of three components: a step penalty, which assigns a small negative reward for every action taken to encourage the agent to reach the goal in fewer steps; a crash penalty, which applies a larger negative reward when the agent collides with an obstacle to promote safe navigation; and a goal reward, which provides a large positive reward upon reaching the target to strongly motivate task completion. Building upon this, the updated reward function introduces three additional terms: a same state penalty to discourage the agent from staying in the same position without progress, an orientation reward to encourage the agent to face and align its direction toward the goal, and a distance reward that provides positive feedback as the agent reduces its distance to the target. The test results show that the design of the reward function greatly impacts finding optimal paths sooner, i.e., in fewer training episodes, and the agent’s overall performance. Our results demonstrate that the updated reward function significantly accelerates learning convergence, leading to optimal path discovery in fewer training episodes and shorter time steps across all tested algorithms and environments. These observations highlight how important reward function shaping is for enhancing an agent’s learning in navigation tasks. The obtained results provide a basis for designing a customized reward function for DRL navigation objectives. The performance validation testing is done w.r.t. improved learning efficiency and faster convergence.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Reward shaping of deep reinforcement learning algorithm for autonomous navigation in a structured environment

  • Anu Priya,
  • Ritu Tiwari,
  • Pooja Agrawal,
  • Sushant Kumar

摘要

Path planning is the essential requirement for autonomous and collision-free movement of a robot agent in the environment. In this paper, the deep Q-network (DQN) and its variants double deep Q-network (DDQN) and duelling double deep Q-network (D3QN) algorithms are considered with a modified reward function for efficient navigation and goal searching system. In the proposed work, initially a base reward function is considered and subsequently an updated reward function is designed to compare the learning’s of an agent’s actions like moving closer to the goal or changing its orientation in reaching the goal to systematically analyze the behavior of the agent. The base reward function consists of three components: a step penalty, which assigns a small negative reward for every action taken to encourage the agent to reach the goal in fewer steps; a crash penalty, which applies a larger negative reward when the agent collides with an obstacle to promote safe navigation; and a goal reward, which provides a large positive reward upon reaching the target to strongly motivate task completion. Building upon this, the updated reward function introduces three additional terms: a same state penalty to discourage the agent from staying in the same position without progress, an orientation reward to encourage the agent to face and align its direction toward the goal, and a distance reward that provides positive feedback as the agent reduces its distance to the target. The test results show that the design of the reward function greatly impacts finding optimal paths sooner, i.e., in fewer training episodes, and the agent’s overall performance. Our results demonstrate that the updated reward function significantly accelerates learning convergence, leading to optimal path discovery in fewer training episodes and shorter time steps across all tested algorithms and environments. These observations highlight how important reward function shaping is for enhancing an agent’s learning in navigation tasks. The obtained results provide a basis for designing a customized reward function for DRL navigation objectives. The performance validation testing is done w.r.t. improved learning efficiency and faster convergence.