In this paper, an ultra-short baseline positioning system-guided (USBL-guided) docking system is proposed to achieve smooth docking guidance for autonomous underwater vehicle (AUV). First, the main challenges of USBL-guided AUV docking are illustrated. Second, to address the issue of discrete USBL positioning samples due to the low positioning frequency and drifting outliers, Least Square Method is applied to select an appropriate initialize sample for USBL filter to prevent the wrong tendency of state prediction caused by the outliers of USBL. Subsequently, we introduce an adaptive exponential weight-improved Kalman Filter that dynamically adjusts the Kalman gain in update process based on the distance between prediction and measurement of state. Furthermore, the derivation process and proof of algorithm are also introduced. The feasibility and efficiency of the improved filter are validated by the comparative experiment carried out in lake trials with the ground truth of dead-reckoning of inertial navigation system (INS). The comparative result shows that the improved KF can efficiently reduce the drifting samples and get a smoother trajectory of AUV.