Scholarlink

MDPs offer a fairly generic and powerful framework to discuss the notion of optimal policies for dynamic systems, in particular when the dynamics are stochastic. However, computing the optimal policy of an MDP can be very difficult due to the curse of dimensionality present in solving the underlying Bellman equations. MPC is a very popular technique for building control policies for complex dynamic systems. Historically, MPC has focused on constraint satisfaction and steering dynamic systems towards a user-defined reference. More recently, EMPC was proposed as a computationally tractable way of building optimal policies for dynamic systems. When stochasticity is present, EMPC is close to the MDP framework. In that context, EMPC can be construed as a tractable heuristic to provide approximate solutions to MDPs. However, there is arguably a knowledge gap in the literature regarding these approximate solutions and the conditions for an EMPC scheme to achieve closed-loop optimality. This chapter aims to clarify this approximation pedagogically, to provide the conditions for EMPC to deliver optimal policies, and to explore some of their consequences.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Economic Model Predictive Control as a Solution to Markov Decision Processes

  • 章节

  • 发表日期：

    2025年

查看全文
  • Dirk Reinhardt,
  • Akhil S. Anand,
  • Shambhuraj Sawant,
  • Sébastien Gros

摘要

MDPs offer a fairly generic and powerful framework to discuss the notion of optimal policies for dynamic systems, in particular when the dynamics are stochastic. However, computing the optimal policy of an MDP can be very difficult due to the curse of dimensionality present in solving the underlying Bellman equations. MPC is a very popular technique for building control policies for complex dynamic systems. Historically, MPC has focused on constraint satisfaction and steering dynamic systems towards a user-defined reference. More recently, EMPC was proposed as a computationally tractable way of building optimal policies for dynamic systems. When stochasticity is present, EMPC is close to the MDP framework. In that context, EMPC can be construed as a tractable heuristic to provide approximate solutions to MDPs. However, there is arguably a knowledge gap in the literature regarding these approximate solutions and the conditions for an EMPC scheme to achieve closed-loop optimality. This chapter aims to clarify this approximation pedagogically, to provide the conditions for EMPC to deliver optimal policies, and to explore some of their consequences.