Purpose <p>Understanding and analyzing the vibrations of a hydroturbine-generator set is necessary for its optimal design. Vibration analysis can also be very useful for implementing maintenance procedures. To obtain useful results from vibration analysis, system modeling is also of particular importance. In this paper, the hydroturbine-generator set is modeled as a couple and various effects such as gyroscopic, arcuate whirl and etc. are considered.</p> Methods <p>A new electromechanical model for the lateral-torsional vibration of a coupled hydroturbine-generator considering is developed. The gyroscopic effect, unbalanced magnetic pull, arcuate whirl, oil film force and seal force are considered in the new mathematical modelling. In the electromechanical model, the equations of the generator and hydroturbine vibration are coupled and should be solved together. The Runge–Kutta numerical method is used to solve governing equations at different rotational speeds and vibration responses are obtained in lateral and torsional directions for hydroturbine and generator rotors. Vibration responses are analyzed through time signal, frequency spectrum, phase plane and Poincare section.</p> Results <p>Result show chaotic behavior in the lateral and torsional vibrations of the hydroturbine and generator rotors. Time signals show periodic behaviors but not exactly repeated. Many and varied combinations of frequencies are observed in the frequency spectrum. Also, the phase planes illustrated chaotic vibrations with different strange attractors. New structured geometries obtained in the Poincare sections which in not reported earlier. The results can be used for the design of hydroturbine-generator set and to monitoring the system condition.</p>