Based on recent developments, unmanned aerial vehicles (UAVs) have emerged as the optimal choice for achieving aerial missions. Most studies have not specifically considered the impact of the wind environment on UAVs, which can result in poor tracking performance. This paper proposes a nonlinear robust tracking controller to settle the UAV’s trajectory tracking problem. The proposed controller is comprised of the active disturbance rejection control (ADRC) and the nonsingular terminal sliding mode control (NTSMC). The method mainly comprises three segments, identified as NTSMC, tracking differentiator (TD), and the extended state observer (ESO). These three parts are respectively employed for reducing system errors, suppressing system chattering, and ensuring system robustness and stability. The robustness and validity of the intended control scheme is demonstrated by the simulation results.