With the rapid advancement of drone technology, its applications in various sectors such as military and civilian domains are becoming increasingly widespread. However, the reliability of the current mainstream Unmanned Aerial Vehicles (UAVs) navigation and positioning algorithm based on Global Navigation Satellite Systems (GNSS) is greatly reduced when the signal is interfered with. To address the navigation and positioning challenges in GNSS-denied scenarios, this paper proposes a fully vision-based autonomous relative navigation algorithm based on interframe and scene matching. The proposed algorithm continuously estimates the vehicle pose through interframe matching, and provides reference position for obtaining the scene matching reference map. By leveraging scene matching, it determines the current positioning and corrects the error accumulation issue inherent in interframe matching. In outdoor flight experiments at 200 m above ground level, the algorithm achieves an average positioning error of 8.36 m, meeting the positioning requirements of drones in denied environments.