The precision of the magnetometer determines the accuracy of the azimuth angle estimation in the Measurement-while-drilling (MWD) system. However, the magnetometer is susceptible to various magnetic interference, which reduces the estimation accuracy of the azimuth angle during the drilling process. To resolve this issue, a dynamic magnetometer calibration algorithm based on Iterative Extended Kalman Filter (IEKF) is proposed in this paper. Firstly, static magnetic interference and dynamic magnetic interference are modelled in the drill tool coordinate system. Then, the static magnetic interference parameters are calibrated based on the ellipsoid fitting calibration algorithm, and online calibration of dynamic magnetic interference is achieved using IEKF. Finally, in a high-precision magnetic environment, the dynamic magnetometer calibration experiment was conducted using the turntable to simulate the drilling process. The experimental results indicate that the accuracy of the azimuth angle has improved by 64.9% compared to the ellipsoid fitting calibration algorithm, with the error decreasing from 2.71° to 0.95°.