The Autonomous Underwater Vehicles (AUVs), as a type of underwater robot, is an important tool for exploration of the ocean. This paper presents a real-time motion planning method for underactuated AUVs based on deep reinforcement learning (DRL) to improve the performance of obstacle avoidance during AUV navigation. Firstly, the motion polar coordinate system of AUV is established, and a path point generator suitable for AUV motion is designed. Then, an AUV motion planning and obstacle avoidance algorithm based on the Actor-Critic (AC) architecture is proposed, along with corresponding state space and reward function designs. Due to the inherent flexibility limitations of underactuated AUVs, significant delays occur in tracking the planned heading, and the discretized output of the neural network can negatively impact stability and path smoothness. To overcome these issues, a path-critic network is introduced. Specifically, a path network is designed to predict the next potential path point, and a critic network assesses both the state and path points. The difference between the historical path of AUV and the path selected by Actor network is used to design residuals for updating the network. Finally, simulation experiments are conducted to validate the performance of the proposed algorithm.