Compared with earth return bipolar DC system, metal return bipolar DC system has different line structure and fault type, which leads to the lack of selectivity and reliability of the existing single-ended fast protection. To solve the above problems, this paper proposes a single end fast protection scheme of flexible DC line based on fault current modal phase plane, which can distinguish the characteristics of different faults and isolate faults and ensure the reliable operation of metal reflux bipolar DC system. In this paper, four different pole-mode transformation matrices are used to calculate the initial value of the fault modulus voltage, and the reasons for the different characteristics of the 0-mode voltages in the initial stage and the transient stage are analyzed. Using TC transformation, the absolute value of 1-mode voltage differential |du1/dt| is used as the starting criterion, 1-mode current variation ∆i1 is used as the direction criterion, and 2-mode and 0 mode current variation ∆i2, ∆i0 is used as the horizontal and vertical coordinates to construct a new modal phase plane to form the pole selection criterion. A single-ended fast protection scheme for metal reflux bipolar DC system based on modulus characteristics is formed. Finally, a four-terminal flexible DC grid model based on frequency dependent line model is built in PSCAD/EMTDC, and the rapidity, selectivity, sensitivity and reliability of the proposed protection scheme are verified by simulation examples.