This study introduces a novel framework for predicting UAV trajectories in GPS-restricted environments using machine learning on features extracted from aerial images. The system employs a downward-facing camera to capture aerial images during flight, extracting key features with the ORB (Oriented FAST and Rotated BRIEF) algorithm to estimate relative movement. The movement data, combined with GPS coordinates obtained during signal availability, is used to train a linear machine learning model (Linear Regression) to estimate latitude and longitude in the event of GPS signal loss. Experimental results in various suburban and mountainous environments demonstrate that the proposed framework accurately estimates UAV positions, showing improved performance in visually diverse areas compared to less diverse environments like mountainous regions. This approach offers a promising solution for enhancing UAV operations in GPS-restricted scenarios.