Traditional linear control methods (e.g., Linear Quadratic Regulator (LQR)) exhibit inherent robustness limitations when addressing the strongly coupled pitch-yaw-roll nonlinear attitude control problem in high-speed UAVs under uncertain operating conditions. This paper proposes an intelligent End-to-end control framework based on Deep Deterministic Policy Gradient (DDPG) that achieves non-decoupled coordinated attitude control through end-to-end policy. The framework constructs an Actor-Critic network in high-dimensional continuous action spaces to directly generate coordinated deflection commands for elevators and rudders. A multimodal compound reward function is designed, incorporating angle-of-attack and bank-angle tracking error penalties, control surface rate constraints, and an exponential convergence acceleration term to balance transient response speed, steady-state accuracy, and actuator smoothness. The algorithm’s generalization capability against model uncertainties is enhanced through delayed target network updates and Ornstein-Uhlenbeck process-driven exploration strategies. Validation is conducted in a comprehensive simulation environment featuring nonlinear aerodynamic coupling terms, ±30% control surface effectiveness variations, and ± 20% aerodynamic coefficient disturbances. Results demonstrate that the proposed data-driven control algorithm significantly outperforms conventional LQR methods in achieving specified closed-loop response performance, exhibiting superior robustness and environmental adaptability.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

End-to-End Control for High-Speed UAVs Using DDPG-Based Deep Reinforcement Learning

  • Haoran Xue,
  • Yong Xi,
  • Guangshan Chen,
  • Feiyi He

摘要

Traditional linear control methods (e.g., Linear Quadratic Regulator (LQR)) exhibit inherent robustness limitations when addressing the strongly coupled pitch-yaw-roll nonlinear attitude control problem in high-speed UAVs under uncertain operating conditions. This paper proposes an intelligent End-to-end control framework based on Deep Deterministic Policy Gradient (DDPG) that achieves non-decoupled coordinated attitude control through end-to-end policy. The framework constructs an Actor-Critic network in high-dimensional continuous action spaces to directly generate coordinated deflection commands for elevators and rudders. A multimodal compound reward function is designed, incorporating angle-of-attack and bank-angle tracking error penalties, control surface rate constraints, and an exponential convergence acceleration term to balance transient response speed, steady-state accuracy, and actuator smoothness. The algorithm’s generalization capability against model uncertainties is enhanced through delayed target network updates and Ornstein-Uhlenbeck process-driven exploration strategies. Validation is conducted in a comprehensive simulation environment featuring nonlinear aerodynamic coupling terms, ±30% control surface effectiveness variations, and ± 20% aerodynamic coefficient disturbances. Results demonstrate that the proposed data-driven control algorithm significantly outperforms conventional LQR methods in achieving specified closed-loop response performance, exhibiting superior robustness and environmental adaptability.