Teleoperation robot systems are commonly used for operations in hazardous, unknown, and specialized environments. As an adaptive control strategy, Reinforcement learning can significantly improve the accuracy and efficiency of master-slave robot collaboration in dynamic environments. To Address the model uncertainty in time-delayed bilateral teleoperation systems and the control accuracy issues arising from the actor-critic (AC) algorithm, this paper proposes an uncertainty-adaptive control method based on reinforcement learning. The proposed approach utilizes a radial basis function neural network for implementation, where the actor network generates control strategies and approximates uncertainties in system dynamics, the critic network evaluates the control process cost. A non-singular fast terminal sliding surface is adopted to guarantee convergence of tracking errors within a predefined time while circumventing the singularity problems inherent in conventional sliding mode control. Subsequently, the stability of the closed-loop system was rigorously examined through Lyapunov function analysis, and the effectiveness and superiority of the proposed method were demonstrated through simulation experiments.