Suppose $(X_{n})_{n \in \mathbb{N}}$ is a discrete time Markov chain with countable state space and transition matrix $P$. Suppose $\mu$ is a stationary distribution, i.e., as a row vector, $\mu = \mu P$. How do I show there is no zero element in the vector $\mu$?
Prove there can be no zero elements in the stationary distribution of an irreducible Markov chain
markov chainsprobabilitystochastic-processes
Best Answer
Suppose the contrary, so $\mu_a = 0$ for some $a$. $\mu$ certainly can't be a zero vector, so there exists $b$ with $\mu_b \not = 0$. Using the definition of an irreducible Markov chain, we know there's some finite sequence $s_0, s_1, \cdots, s_n$ satisfying $s_0 = b$, $s_n = a$, and $P_{s_{k+1} s_k} \neq 0$ for all $0 \le k < n$.
We then have $(P \mu)_{s_1} = \sum_k P_{s_1 k} \mu_k \ge P_{s_1 s_0} \mu_{s_0} > 0$. By basically the same argument, $(P^2 \mu)_{s_2} > 0$, and inductively we find $(P^n \mu)_{s_n} > 0$. On the other hand we know $\mu$ is a stationary distribution, so $P^n \mu = \mu$ and thus $(P^n \mu)_{s_n} = \mu_{s_n} = \mu_a = 0$. That's a contradiction, so $\mu$ must have all entries nonzero after all.