markov chainsstochastic-processes
Suppose I have a transition matrix for a Markov chain, which has states {1,2,3}, such as:
\begin{bmatrix}0&1&0\\0&0&1\\0&0&1\end{bmatrix}
I wonder if this chain is reducible or not? Moreover, from my textbook, it says the communication class will partition the states of the Markov chain. However, in this case, only state 3 communicate to itself, while other states are transient. How does the communication class partition all states, since the only communication class is {3}?
Indeed this Markov chain is reducible, with two communicating classes, and the communicating class {1} is closed while the communicating class {2,3} is not. Every stationary distribution of a Markov chain is concentrated on the closed communicating classes, in the present case, only state 1 is in a closed communicating class. This proves without computation that the stationary distribution is unique and is the Dirac distribution on state 1.
Here's a graph depiction of the Markov chain (including all possible transitions, but ignoring loops [transitions from a state to itself]).
So there are three equivalence classes (0 and 6 should be together, and 5 was missing from the original attempt). Also states in $\{1,3,5\}$ should be recurrent.
Best Answer
This case depends somewhat on precisely how you define communicating class, but the most mathematically clean way to set up the definitions is to say that $i$ communicates with $j$ if there exist nonnegative integers $m=m(i,j),n=n(i,j)$ such that $P^m_{ij}>0$ and $P^n_{ji}>0$. Then taking $m=n=0$ tells you that each state communicates with itself.
If you use this definition, then the communicating classes of this chain are just the three singleton sets. If you instead require $m,n$ to be positive, then the only communicating class is $\{ 3 \}$ and the other states are simply not in any communicating class.