This paper proposes a relative entropy planning method for cruise trajectory optimization of air-breathing hypersonic aircraft subject to constraints such as no-fly zones. The method is grounded in the Probably Approximately Correct (PAC)-Bayes theory, which enables deriving high-probability upper bounds on the expected cost of (stochastic) control policies in new scenarios. Then the policy learning algorithms is proposed to minimize this upper bound and Relative Entropy Programming (REP) is utilized to solve the corresponding optimization problem. Finally, numerical demonstrations are provided to illustrate the effectiveness and efficiency of the proposed method for onboard applications, and the method has strong generalization performance in new scenarios.