This paper investigates a robust adaptive controller using deep reinforcement learning for hypersonic flight vehicles with aerodynamic uncertainties. Based on the subsystems of angle of attack, sideslip angle and bank angle, an active disturbance rejection controller is designed to obtain the deflections of left elevator, right elevator and rudder, where the extended state observer (ESO) is utilized to estimate aerodynamic uncertainty. More specifically, deep reinforcement learning strategy is employed to achieve the adaptive adjustment of ESO bandwidth. Deep neural networks (NNs) are trained offline under multiple flight conditions, and well-trained NNs are deployed online to generate effective observer parameters. Based on the simulation results with random parameter perturbation, the proposed design exhibits excellent performance in tracking and learning accuracy.