In this paper, we implement control over the Markov chain, which causes it to become a time-inhomogeneous Markov chain. One of its most prominent features is that the transition rate matrix varies over time, which makes the time-inhomogeneous Markov chain with rate control can be more congruent with practical situations. Nevertheless, it is precisely this characteristic that renders the analysis of it more difficult. Thus we present a framework to analyze its asymptotic behavior based on the corresponding probability equations. In particular, by deriving the convergence of its solution, we can get the steady-state distribution of the underlying time-inhomogeneous Markov chain. In addition, we also provide an example to demonstrate this framework for substantiating its feasibility. Finally, we conducted numerical experiments and the results indicate that it can reach equilibrium as time approaches infinity, which is consistent with the convergence of the solution to its probability equations.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Time-Inhomogeneous Markov Chain with Rate Control

  • Junteng Liu,
  • Xiang Wei,
  • Jie Ding

摘要

In this paper, we implement control over the Markov chain, which causes it to become a time-inhomogeneous Markov chain. One of its most prominent features is that the transition rate matrix varies over time, which makes the time-inhomogeneous Markov chain with rate control can be more congruent with practical situations. Nevertheless, it is precisely this characteristic that renders the analysis of it more difficult. Thus we present a framework to analyze its asymptotic behavior based on the corresponding probability equations. In particular, by deriving the convergence of its solution, we can get the steady-state distribution of the underlying time-inhomogeneous Markov chain. In addition, we also provide an example to demonstrate this framework for substantiating its feasibility. Finally, we conducted numerical experiments and the results indicate that it can reach equilibrium as time approaches infinity, which is consistent with the convergence of the solution to its probability equations.