Mobile machining robots can flexibly change their working stations, providing a new solution for drilling on large component. Since there is no fixed positional relationship between the robot and the workpiece, reference holes can be pre-made on the workpiece, and external measuring equipment can be used to locate these reference holes to determine the positional relationship. The measuring accuracy of the reference holes directly affects the drilling accuracy. As a low-cost solution, the high-resolution binocular vision method can obtain three-dimensional coordinates of the reference hole positions. However, fast and accurate measurement of reference holes in high-resolution images presents technical challenges. In order to improve the drilling efficiency and accuracy of robots, this paper proposes a two-step measuring method of reference holes. Neural networks are trained for the prediction of reference holes in the images, effectively eliminating background interference in high-resolution images and enhancing the efficiency of measurement. Feature operators and morphological processing algorithms are designed to optimize the edges of holes, improving the effect of ellipse fitting and ensuring the accuracy of measurement. This method achieves an average reference hole positioning accuracy better than 0.04 mm. Finally, the proposed scheme is integrated into a mobile hybrid machining robot to form an automatic drilling system, achieving an average drilling accuracy of 0.075 mm.