Policy Gradient Algorithms
摘要
The policy gradient method is another commonly used method in deep reinforcement learning. Unlike value-based methods, it directly updates the policy by calculating the gradient of the expected total reward with respect to the policy parameters, aiming to solve some shortcomings of value-based methods. This chapter is to introduce several policy gradient methods. Firstly, the definition of policy gradient is given. Secondly, the basic form of policy gradient is deduced. Finally, three types of policy gradient methods are presented, including actor-critic-based policy gradient methods, deep deterministic policy gradient methods, and asynchronous reinforcement learning methods. In actor-critic-based policy gradient methods, the actor updates actions based on the policy gradient algorithm, and the critic evaluates actions based on the value function method. In deep deterministic policy gradient, deterministic policy gradient (DPG) algorithm and deep deterministic policy gradient (DDPG) algorithm are introduced. In asynchronous reinforcement learning methods, asynchronous advantage actor-critic (A3C) is analyzed.