Scholarlink

In reinforcement learning, reward shaping is an efficient way to augment the reward signal, so to guide the learning process of an agent. A well-known reward shaping framework is the potential-based reward shaping (PBRS) framework, which uses a so-called potential function to guarantee the policy invariance after reward shaping, to prevent undesirable behavior. Different from using a predefined potential function in many works, [3] proposed a novel adaptive potential function (APF) method to learn the potential function concurrently with the RL training from the agent’s training history. However, the APF method was only deployed and evaluated in small discrete environments. This paper bridges the gap by adapting the APF method in robotics, a typical continuous scenario. We apply the APF method with the Deep Deterministic Policy Gradient (DDPG) algorithm to form a new APF-DDPG algorithm. To evaluate our method, we deploy the APF-DDPG to control a Baxtor robot for a series of reaching tasks in both simulations and the real world. The experimental results show that the APF-DDPG algorithm significantly outperforms the baseline DDPG algorithm. The code is available at https://github.com/yfchenShirley/APF_DDPG.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Boosting Reinforcement Learning Algorithms in Continuous Robotic Reaching Tasks Using Adaptive Potential Functions

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Yifei Chen,
  • Lambert Schomaker,
  • Francisco Cruz

摘要

In reinforcement learning, reward shaping is an efficient way to augment the reward signal, so to guide the learning process of an agent. A well-known reward shaping framework is the potential-based reward shaping (PBRS) framework, which uses a so-called potential function to guarantee the policy invariance after reward shaping, to prevent undesirable behavior. Different from using a predefined potential function in many works, [3] proposed a novel adaptive potential function (APF) method to learn the potential function concurrently with the RL training from the agent’s training history. However, the APF method was only deployed and evaluated in small discrete environments. This paper bridges the gap by adapting the APF method in robotics, a typical continuous scenario. We apply the APF method with the Deep Deterministic Policy Gradient (DDPG) algorithm to form a new APF-DDPG algorithm. To evaluate our method, we deploy the APF-DDPG to control a Baxtor robot for a series of reaching tasks in both simulations and the real world. The experimental results show that the APF-DDPG algorithm significantly outperforms the baseline DDPG algorithm. The code is available at https://github.com/yfchenShirley/APF_DDPG.