Scholarlink

<p>This paper investigates large deviation asymptotics for empirical measures of Markov decision processes on compact space. Building on results in risk-sensitive control, we derive a large deviation upper bound for state–action frequencies uniformly over initial states and policies. Zero points of the corresponding rate function are used to characterize accumulation points of empirical measures and their expected values. For the state frequencies, we also obtain a large deviation lower bound, thereby establishing a controlled version of the large deviation principle. Results for the empirical means of rewards are also presented. As an application, we establish the existence of an <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(\epsilon \)</EquationSource> <EquationSource Format="MATHML"><math> <mi>ϵ</mi> </math></EquationSource> </InlineEquation>-optimal randomized stationary policy for a large deviation control problem.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Large Deviations and Optimal Control for Markov Decision Processes

  • 期刊论文

  • 发表日期：

    2026年5月6日

查看全文
  • Xiaoyang Lu,
  • Jinwen Chen

摘要

This paper investigates large deviation asymptotics for empirical measures of Markov decision processes on compact space. Building on results in risk-sensitive control, we derive a large deviation upper bound for state–action frequencies uniformly over initial states and policies. Zero points of the corresponding rate function are used to characterize accumulation points of empirical measures and their expected values. For the state frequencies, we also obtain a large deviation lower bound, thereby establishing a controlled version of the large deviation principle. Results for the empirical means of rewards are also presented. As an application, we establish the existence of an \(\epsilon \) ϵ -optimal randomized stationary policy for a large deviation control problem.