Solved – Classification of states in Markov Chain

ergodicmarkov-processself-studystochastic-processes

Question

Consider the following transition matrix:

 P= 
 0     0     1     0     0     0 
 0     0     0     0     0     1
 0     0     0     0     1     0
1/4   1/4    0    1/2    0     0
 1     0     0     0     0     0
 0    1/3    0     0     0    2/3

a) Which states are transient?
b) Which states are recurrent?
c) Identify all closed sets of states.
d) Is this chain ergodic?

Dear friends ,

I have thought the states are as follows,
{1,5} {0,2,4} recurrent since they communicate with each other
{3} transient

and since state 3 does not communicate with other states, it is not an ergodic Mc

Am I correct?

Thank you..

Best Answer

Let the state space of the Markov Chain be $S=\{1,2,3,4,5,6\}$. Now draw the state transition diagram.

enter image description here

(a). From the figure, we observe that $\{4\}$, and $\{6\}$ form non-closed communicating classes. State $2$ does not communicate even with itself and such a state is called a non-return state. Hence, the states 2, 4 and 6 are transient.

(b)&(c). The class $\{1,3,5\}$ is a closed-communicating class. Hence, states 1, 3 and 5 are recurrent states.

(c). There is only one closed-communicating class, $\{1,3,5\}$.

(d). As the chain is not an irreducible Markov Chain, it is not an ergodic chain.

Related Question