Accurate relative distance estimation provides unique information for the docking phase of autonomous aerial refueling (AAR), but most monocular vision based distance estimation methods suffer from increasing error along with the growing distance. To address this problem, this paper proposes a robust drogue distance estimation method based on YOLOv3 and Linear Regression (LR) algorithm to obtain more precise distance prediction. To be specific, this method achieves a lower relative distance error within 20m work-range on our simulation image datasets. Furthermore, it also has been deployed on Nvidia Jetson AGX Xavier to infer camera video stream with 30 fps speed, which proves the effectiveness of real-time drogue distance estimation.