The Abnormal cycloidal gear (ACG) reducer is a new 2 K-H internal meshing reducer. A comprehensive modification method along the tooth profile and tooth lead is adopted to tackle the problem of stress concentration and meshing impact during the operating stage in ACG reducer. Structural characteristics of ACG reducer and its transmission tooth are designed, and mathematical model of ACG gear modification is established. Based on the loaded tooth contact analysis, the influence of modification on the contact pattern, contact stress, and contact force are studied. The results demonstrate that the contact pattern of the modified gear is elliptical, and the tooth contact stress field of the tooth surface is significantly improved, which solves the problem of meshing impact and stress concentration, reduces the sensitivity of the gear pair to the axial installation error and signifycantly improves the transmission performance.