Air refueling technology can effectively enhance the endurance of the aircraft. It is a multiplier of air power, and plays a vital role in improving the combat capability of manned aircraft and UAV. However, it is difficult and dangerous to do air refueling, as the relative position of the refueling cone sleeve cannot be accurately obtained in real time through the visual perception of the pilot. In this paper, the scale-adaptive cone-set target tracking algorithm is researched, the scale adaptive correlation tracking algorithm framework based on feature point detection is proposed by analyzing the problems of scale fixation and inability to judge tracking failure, and the feature point detection method based on convolution neural network is studied to realize the high precision, high robustness and high real-time of cone target tracking. The experiments show that the scale adaptive tracking algorithm based on feature point detection greatly improves the tracking accuracy compared with the traditional related tracking algorithm, and ensures the excellent real-time effect of related tracking.