A Markov process is a stochastic process used to model the events, where the system moves between states and the probability of transitioning from one state to the next state depends solely on the current state, with no influence from the previous visited states. Markov processes are categorized based on the nature of their state and parameter space. When the state space is discrete, the process is called a Markov chain (MC). If both the state and parameter space are discrete, it is specifically known as a discrete-time Markov chain (DTMC). In this chapter, the definition and basic properties of DTMC are presented. Also, limiting distribution, stationary distribution and steady-state analysis of the DTMCs are also considered. Moreover, this chapter and Chap.  3 both are devoted to an elaboration of the basic concepts and methods of DTMC.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Discrete-Time Markov Chains—Part I

  • Dharmaraja Selvamuthu

摘要

A Markov process is a stochastic process used to model the events, where the system moves between states and the probability of transitioning from one state to the next state depends solely on the current state, with no influence from the previous visited states. Markov processes are categorized based on the nature of their state and parameter space. When the state space is discrete, the process is called a Markov chain (MC). If both the state and parameter space are discrete, it is specifically known as a discrete-time Markov chain (DTMC). In this chapter, the definition and basic properties of DTMC are presented. Also, limiting distribution, stationary distribution and steady-state analysis of the DTMCs are also considered. Moreover, this chapter and Chap.  3 both are devoted to an elaboration of the basic concepts and methods of DTMC.