Cable traveling wave modes propagation coefficients can reflect the propagation characteristics of traveling wave, and the study of its frequency characteristics can provide theoretical basis for cable fault identification fault and localization, traveling wave protection calibration and so on. In this paper, through the establishment of the frequency-variable parameter model of the cable, simulation to obtain the impedance-conductance matrix of the cable; based on the decoupling method of phase-mode transformation, we obtained the frequency characteristic curve of the six modes propagation coefficients of the cable line, analyzed in detail the frequency characteristics of modes propagation coefficients in the full frequency band, and verified the correctness of the frequency characteristics of modes propagation coefficients through the analysis of the modes travelling wave of the cable fault current.