The hybrid inertial navigation system (HINS) incorporates stabilized platforms, strapdown algorithms, and rotation modulation techniques to further enhance navigation performance. However, the rotating structure brings attitude loss, which limits the further promotion of HINS. Current solutions mostly rely on compensating for the INS error parameters with little reference to the encoder measurement errors. As a vital component of the HINS gimbal, measurement error of circular grating ruler is susceptible to change even after calibration, which indirectly affects azimuth accuracy. To address this problem, this paper proposes an azimuthal oscillation improvement method based on grating error identification. First, the alignment and navigation mechanism of the dual-axis HINS is presented and the effects caused by the grating angle error are analyzed. Then, the error is identified by z-gyro angular velocity and inner grating angle, while a Fourier series compensation model including temperature parameters is constructed. Finally, the feasibility and significance of the method are verified by a static base experiment.