Scholarlink

Abstract <p>Hydrothermal power-system management involves high-dimensional uncertainty (curse of dimensionality), challenging traditional stochastic programming. Stochastic dynamic programming (SDP) and scalable variants remain robust industrial baselines but face rising computational strain as grids add flexibility and variability. This paper presents an enhanced reinforcement learning (RL) framework for long-horizon hydrothermal dispatch, extending tabular approaches via a scalable <i>Gymnasium</i>-based representation of the Uruguayan power system. Alongside a tabular Q-learning baseline, it introduces a proximal policy optimization (PPO) agent for continuous state-action control, representing reservoir volumes and turbine releases without discretization. Optuna-based hyperparameter optimization is applied to both methods, and stochastic results use multiseed ensemble statistics. Performance is evaluated under deterministic inflows and under stochastic settings using a nonhomogeneous Markov inflow model (in-sample) and century-scale historical chronicles (out-of-sample). Within the adopted environment, PPO achieves a 3.96% lower cost than Q-learning (2328.17 vs 2424.28 MUSD), and both are close to the MOP reference (2463.64 MUSD). Out-of-sample historically, Q-learning is within 1.29% of MOP (2761.25 vs 2726.07 MUSD), while PPO attains 2645.01 MUSD within the adopted simplified environment. The purpose of this study is methodological rather than operational: it does not aim to provide a production-ready solution to the full hydrothermal scheduling problem, but to assess whether RL agents can learn stable, robust, and physically meaningful long-horizon dispatch policies in a controlled simplified environment. Comparisons must consider the model gap: MOP includes head-dependent hydropower physics, whereas the RL environment assumes a constant energetic conversion coefficient, affecting low-head costs. Accordingly, the reported benchmarks should be interpreted as evidence of convergence and policy coherence under transparent simplifying assumptions, rather than as a claim of full operational equivalence or deployment readiness.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A Reinforcement Learning Approach for Long-term Hydrothermal Dispatch

  • 期刊论文

  • 发表日期：

    2026年6月23日

查看全文
  • Florencia Roque,
  • Matías Rama,
  • Ignacio Salas,
  • Mónica Carle,
  • Magdalena Irurtia,
  • Rodrigo Porteiro

摘要

Abstract

Hydrothermal power-system management involves high-dimensional uncertainty (curse of dimensionality), challenging traditional stochastic programming. Stochastic dynamic programming (SDP) and scalable variants remain robust industrial baselines but face rising computational strain as grids add flexibility and variability. This paper presents an enhanced reinforcement learning (RL) framework for long-horizon hydrothermal dispatch, extending tabular approaches via a scalable Gymnasium-based representation of the Uruguayan power system. Alongside a tabular Q-learning baseline, it introduces a proximal policy optimization (PPO) agent for continuous state-action control, representing reservoir volumes and turbine releases without discretization. Optuna-based hyperparameter optimization is applied to both methods, and stochastic results use multiseed ensemble statistics. Performance is evaluated under deterministic inflows and under stochastic settings using a nonhomogeneous Markov inflow model (in-sample) and century-scale historical chronicles (out-of-sample). Within the adopted environment, PPO achieves a 3.96% lower cost than Q-learning (2328.17 vs 2424.28 MUSD), and both are close to the MOP reference (2463.64 MUSD). Out-of-sample historically, Q-learning is within 1.29% of MOP (2761.25 vs 2726.07 MUSD), while PPO attains 2645.01 MUSD within the adopted simplified environment. The purpose of this study is methodological rather than operational: it does not aim to provide a production-ready solution to the full hydrothermal scheduling problem, but to assess whether RL agents can learn stable, robust, and physically meaningful long-horizon dispatch policies in a controlled simplified environment. Comparisons must consider the model gap: MOP includes head-dependent hydropower physics, whereas the RL environment assumes a constant energetic conversion coefficient, affecting low-head costs. Accordingly, the reported benchmarks should be interpreted as evidence of convergence and policy coherence under transparent simplifying assumptions, rather than as a claim of full operational equivalence or deployment readiness.