Is it possible for a reducible markov chain to have a unique stationary distribution. Consider e.g. the markov chain with transition matrix below
$$A=
\begin{pmatrix}
1 & 0 & 0 \\\
0.2 & 0.7 & 0.1 \\\
0.3 & 0.3 & 0.4
\end{pmatrix}$$
I believe A is a reducible markov chain ({1} and {2,3} are two distinct classes of states). Once we visit state 1 we are stuck with state 1. However if we try to calculate its unique distribution it comes as pi_s = [1 0 0]. And if we start with any arbitrary probability distribution, after a while it seems we will converge to pi_s. I am not sure am I missing something?
Best Answer
Indeed this Markov chain is reducible, with two communicating classes, and the communicating class {1} is closed while the communicating class {2,3} is not. Every stationary distribution of a Markov chain is concentrated on the closed communicating classes, in the present case, only state 1 is in a closed communicating class. This proves without computation that the stationary distribution is unique and is the Dirac distribution on state 1.