<p>The ship propulsion shaft-bearing system plays a crucial role in the ship’s power system, and the friction-induced self-excited vibrations within the system significantly affect the stability of ship operation. This paper establishes a four-degree-of-freedom coupled dynamic model for the ship propulsion shaft system on the basis of modal-coupling theory. This approach explored the variation trends of system parameters on modal-coupling stability and analyzed the time-domain characteristics as speed changes under varying parameters. The findings indicate that the critical friction coefficient is closely related to the minimum critical friction coefficient; furthermore, the resonant frequencies of the system gradually converge as the friction coefficient increases. It indicates that the possibility of experiencing friction self-excited vibration is reduced under conditions of high stiffness, high damping, and low deflection angle. A comparison between modal-coupling theory and time-domain analysis further elucidates the complex effects of friction-induced self-excited vibrations on ship stability.</p>