<p>Energy management in hydrogen fuel cell hybrid unmanned aerial vehicles (UAVs) must balance fuel economy, charge-sustaining operation, health-state preservation, and online feasibility under time-varying propulsion demand. Equivalent Consumption Minimization Strategy (ECMS) has a clear physical structure, but its performance is sensitive to the equivalence factor (EF), whereas pure end-to-end deep reinforcement learning (DRL) may suffer from weak interpretability and larger run-to-run variability. This paper proposes a hierarchical Twin Delayed Deep Deterministic Policy Gradient–Equivalent Consumption Minimization Strategy (TD3-ECMS) framework, in which an upper-layer TD3 agent adapts the EF online and a lower-layer ECMS performs instantaneous power-split optimization. Fuzzy logic control (FLC), fixed-EF ECMS, pure end-to-end TD3, and TD3-ECMS are compared to separate the contribution of ECMS-based optimization from that of TD3-based online EF adaptation. Under the standard nominal mission, FLC obtains an actual hydrogen consumption of 2024.3 g, an equivalent hydrogen consumption of 2013.5 g, and a terminal SOC deviation of 0.0320, while fixed-EF ECMS obtains 1939.0 g, 1936.3 g, and 0.0079, respectively. Over 30 independent random seeds, TD3-ECMS reduces the mean equivalent hydrogen consumption from <InlineEquation ID="IEq1"> <EquationSource Format="TEX">$1995.90\pm 49.64$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mn>1995.90</mn> <mo>±</mo> <mn>49.64</mn> </mrow> </math></EquationSource> </InlineEquation> g to <InlineEquation ID="IEq2"> <EquationSource Format="TEX">$1931.17\pm 2.56$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mn>1931.17</mn> <mo>±</mo> <mn>2.56</mn> </mrow> </math></EquationSource> </InlineEquation> g and decreases the terminal SOC error from <InlineEquation ID="IEq3"> <EquationSource Format="TEX">$9.60\pm 12.33\%$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mn>9.60</mn> <mo>±</mo> <mn>12.33</mn> <mo>%</mo> </mrow> </math></EquationSource> </InlineEquation> to <InlineEquation ID="IEq4"> <EquationSource Format="TEX">$1.45\pm 0.98\%$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mn>1.45</mn> <mo>±</mo> <mn>0.98</mn> <mo>%</mo> </mrow> </math></EquationSource> </InlineEquation> relative to pure TD3, with Mann–Whitney U test results of <InlineEquation ID="IEq5"> <EquationSource Format="TEX">$p<0.001$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi>p</mi> <mo><</mo> <mn>0.001</mn> </mrow> </math></EquationSource> </InlineEquation> for the main learning-based comparisons. Additional short-duration, long-duration, wind-disturbed, and modified-load profiles are used for robustness checks. The thermal and degradation models are also explicitly described, including the fuel cell water cooling term, altitude-dependent ambient temperature boundary, and battery heat dissipation model. A controller-only timing benchmark shows that the maximum online decision time of TD3-ECMS is 9.561 ms, accounting for <InlineEquation ID="IEq6"> <EquationSource Format="TEX">$0.956\%$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mn>0.956</mn> <mo>%</mo> </mrow> </math></EquationSource> </InlineEquation> of the 1 s lower-layer control period. These results indicate that TD3-ECMS substantially improves the statistical robustness and charge-sustaining behavior of pure TD3, while achieving fuel economy and model-based health-state behavior comparable to or slightly better than the calibrated fixed-EF ECMS baseline under most tested profiles.</p>

Data-driven energy management of hydrogen fuel cell hybrid unmanned aerial vehicles via a hierarchical TD3-ECMS framework

期刊论文
发表日期：

2026年6月22日

查看全文

Haobo Zhou,
Hongyi Zhang,
Zichun Lin,
Weichao Zhuang

摘要

Energy management in hydrogen fuel cell hybrid unmanned aerial vehicles (UAVs) must balance fuel economy, charge-sustaining operation, health-state preservation, and online feasibility under time-varying propulsion demand. Equivalent Consumption Minimization Strategy (ECMS) has a clear physical structure, but its performance is sensitive to the equivalence factor (EF), whereas pure end-to-end deep reinforcement learning (DRL) may suffer from weak interpretability and larger run-to-run variability. This paper proposes a hierarchical Twin Delayed Deep Deterministic Policy Gradient–Equivalent Consumption Minimization Strategy (TD3-ECMS) framework, in which an upper-layer TD3 agent adapts the EF online and a lower-layer ECMS performs instantaneous power-split optimization. Fuzzy logic control (FLC), fixed-EF ECMS, pure end-to-end TD3, and TD3-ECMS are compared to separate the contribution of ECMS-based optimization from that of TD3-based online EF adaptation. Under the standard nominal mission, FLC obtains an actual hydrogen consumption of 2024.3 g, an equivalent hydrogen consumption of 2013.5 g, and a terminal SOC deviation of 0.0320, while fixed-EF ECMS obtains 1939.0 g, 1936.3 g, and 0.0079, respectively. Over 30 independent random seeds, TD3-ECMS reduces the mean equivalent hydrogen consumption from $1995.90\pm 49.64$ $1995.90 \pm 49.64$ g to $1931.17\pm 2.56$ $1931.17 \pm 2.56$ g and decreases the terminal SOC error from $9.60\pm 12.33\%$ $9.60 \pm 12.33 %$ to $1.45\pm 0.98\%$ $1.45 \pm 0.98 %$ relative to pure TD3, with Mann–Whitney U test results of $p<0.001$ $p < 0.001$ for the main learning-based comparisons. Additional short-duration, long-duration, wind-disturbed, and modified-load profiles are used for robustness checks. The thermal and degradation models are also explicitly described, including the fuel cell water cooling term, altitude-dependent ambient temperature boundary, and battery heat dissipation model. A controller-only timing benchmark shows that the maximum online decision time of TD3-ECMS is 9.561 ms, accounting for $0.956\%$ $0.956 %$ of the 1 s lower-layer control period. These results indicate that TD3-ECMS substantially improves the statistical robustness and charge-sustaining behavior of pure TD3, while achieving fuel economy and model-based health-state behavior comparable to or slightly better than the calibrated fixed-EF ECMS baseline under most tested profiles.