Purpose <p>Operating large-scale offshore wind turbines (OWTs) always experience a combination of external aerodynamic and hydrodynamic loadings together with self-excitations from blade rotation. This leads to complex dynamic behaviors of the OWT with multiple vibration modes which cannot be fully simulated with conventional mode assumption method. This paper proposes an innovative approach to model such behaviors with multi-body dynamics which enables the simulation of higher-order modes in the monopile OWT structure.</p> Methods <p>Dynamical finite element method is adopted to formulate the local consistent dynamic matrices of Timoshenko beam in the OWT considering shear deformations of tower and blade. The global dynamic matrices of the OWT structure are then obtained from coordinate transformation between the blades, driveshaft and nacelle servo controllers. The Blade Element Momentum (BEM) theorem with rotational wind-sampling is utilized to model the aerodynamic loadings, and Morrison equation is employed to simulate the hydrodynamic loadings.</p> Results and Conclusion <p>The proposed approach is noted to have higher interpolating accuracy of internal forces and stresses of the OWT due to the smaller interpolation interval in the blade and tower elements. The computation accuracies on the natural frequencies and dynamic responses are noted quite satisfactory even with consideration of the higher-order vibrations of the structure.</p>