[Math] Why is this infinite-state-space Markov chain positive recurrent

markov chainsmarkov-processstochastic-processes

Given the following transition matrix for a Markov chain, how can I see that the chain is positive recurrent? I want to convince myself that the chain has a limiting distribution, and the chain is clearly aperiodic and irreducible, so all I need now is to show that the chain is positive recurrent.

$ \left( \begin{array}
qq & p & 0&0&0&0&… \\
q & 0 & p&0&0&0&… \\
q & 0 & 0&p&0&0&… \\
q & 0 & 0&0&p&0&… \\
… & … \\
… & … \\ \end{array} \right)$

Best Answer

The stationary distribution is given by $\pi_k:=qp^{k-1}$ (in the non degenerate case, that is $pq\neq 0$). In this case, since the chain is irreducible, the states are all positive recurrent.

Related Question