This paper explores an adaptive fault-tolerant control method for unmanned surface vessels (USVs) equipped with multiple thrusters, which is based on safe reinforcement learning. The objective of the proposed method is to prevent the unexpected behavior of the USV and the subsequent accident due to the failure of one or more thrusters. Firstly, a framework of safe deep reinforcement learning for controlling USV is proposed; secondly, a disturbance network that impacts certain thrusters during specific scenarios is introduced to the environment module, which could simulate various failures on thrusters; thirdly, the control strategies of the USV are trained to track desired speed and course, and a safety constraint function is introduced to ensure the safety of USV during changes on speed and course. Lastly, the Line of Sight (LOS) guidance principle is applied to guide the USV to the designed trajectory. To validate the proposed method, the USV with four thrusters is employed in simulation, and scenarios with full-functional thrusters, one-thruster failure, and two-thruster failure are simulated. Compared with PID controller designed by pole placement method, the proposed method would help the USV track the trajectory when thrusters fail, ensuring both the safety and stability of the USV.