Scholarlink

<p>This paper studies multi-objective infinite-horizon discounted continuous-time Markov decision processes (CTMDPs) with distinct discount rates across the objectives. We start the analysis through the weighted sum method. A counterexample demonstrates the absence of Pareto-optimal stationary policies, contrasting with the classic discounted CTMDP results. Inspired by the idea for finite-horizon scenarios, we derive the Hamilton-Jacobi-Bellman equation with time as the differential variable for the infinite-horizon weighted sum optimization. To solve the new equation, we introduce a novel dynamic programming operator along with a carefully constructed initial function. Our value iteration method establishes the existence of weighted-optimal Markov deterministic policies, thereby ensuring the existence of Pareto-optimal policies. Moreover, we prove weak Pareto sufficiency of weighted-optimal policies by the occupation measure technique. Finally, we characterize the structure of <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(\varepsilon \)</EquationSource> <EquationSource Format="MATHML"><math> <mi>ε</mi> </math></EquationSource> </InlineEquation>-optimal policies—they are ultimately stationary under specific conditions, and provide an algorithm to calculate <InlineEquation ID="IEq2"> <EquationSource Format="TEX">\(\varepsilon \)</EquationSource> <EquationSource Format="MATHML"><math> <mi>ε</mi> </math></EquationSource> </InlineEquation>-optimal policies for finite-state cases.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Multi-Objective Discounted Continuous-Time Markov Decision Processes

  • 期刊论文

  • 发表日期：

    2026年6月27日

查看全文
  • Zhuoxin Chen,
  • Yonghui Huang

摘要

This paper studies multi-objective infinite-horizon discounted continuous-time Markov decision processes (CTMDPs) with distinct discount rates across the objectives. We start the analysis through the weighted sum method. A counterexample demonstrates the absence of Pareto-optimal stationary policies, contrasting with the classic discounted CTMDP results. Inspired by the idea for finite-horizon scenarios, we derive the Hamilton-Jacobi-Bellman equation with time as the differential variable for the infinite-horizon weighted sum optimization. To solve the new equation, we introduce a novel dynamic programming operator along with a carefully constructed initial function. Our value iteration method establishes the existence of weighted-optimal Markov deterministic policies, thereby ensuring the existence of Pareto-optimal policies. Moreover, we prove weak Pareto sufficiency of weighted-optimal policies by the occupation measure technique. Finally, we characterize the structure of \(\varepsilon \) ε -optimal policies—they are ultimately stationary under specific conditions, and provide an algorithm to calculate \(\varepsilon \) ε -optimal policies for finite-state cases.