<p>This study uses a reinforcement learning (RL) algorithm to address the trajectory tracking control problem for a robotic manipulator subject to disturbances. A disturbance observer is developed to estimate and counteract external disturbances and model inaccuracies, thereby enhancing the manipulator’s control precision and disturbance rejection capability. A tracking controller is devised to improve tracking performance while maintaining control costs by leveraging the double Q-learning algorithm within reinforcement learning. Utilizing double Q-learning mitigates the issue of Q value overestimation encountered in traditional Q-learning approaches. This method significantly improves the robustness and adaptive ability of the control strategy by introducing a double Q network structure. It provides a new solution for accurate trajectory tracking of the robotic manipulator in unknown and changing environments. At the same time, the robotic manipulator can learn the optimal control strategy more quickly in the face of external disturbance and system uncertainty to achieve better trajectory tracking performance. Simulation and experiment results affirm the efficacy of the proposed control strategy, demonstrating superior trajectory tracking performance and disturbance attenuation capabilities for the manipulator system.</p>