Computing $\lim_n P(X_n=A|X_0=C)$ of a Markov chain

Question

Suppose that $(X_n)_{n\geq1}$ is markov chain with state space $S=\{A,B,C,D,E \}$ with the following transition matrix
$$ P = \left( \begin{matrix} 0.6 & 0.4 & 0 & 0 & 0 \\ 0.3 & 0.7 & 0 & 0 & 0 \\ 0.2 & 0 & 0.4 & 0 & 0.4 \\ 0.2 & 0.2 & 0.2 & 0.2 & 0.2 \\ 0 & 0 & 0 & 0 & 1 \end{matrix} \right) $$

I want to find the following probability $\lim_nP(X_n=A|X_0=C)$

I found that $C_A=\{A,B\}$ is a communicating class with both $A,B$ recurrent states.
There is only one way to get from state $C$ to state $A$ and that happens with probability $p_{AB}=0.2$.
I also know that once we hit the set $C_A$ this can be seen as an irreducible markov chain on state space ${A,B}$ and the one step probabilities will converge to $\pi_A=\frac{3}{7},\pi_B=\frac{4}{7}$.
How to combine this to find $\lim_nP(X_n=A|X_0=C)$?

Answer 1

Let $\tau=\inf\{n>0: X_n\ne C\mid X_0=C\}$. First we compute \begin{align} \mathbb P(X_\tau = A) &= \frac{P_{CA}}{P_{CA}+P_{CE}}\\ &= \frac{\frac15}{\frac15+\frac25}\\ &= \frac13 \end{align} (we can ignore the self-transitions from state $C$ to itself). Then, conditioned on the event $\{X_\tau=A\}$, we have $\{X_{\tau+n} : n=0,1,\ldots\}$ as an irreducible Markov chain on $\{A,B\}$, with transition matrix given by the submatrix obtained by taking the rows and columns of $P$ corresponding to states $A$ and $B$. You have already computed the stationary distribution for this Markov chain - so the limiting probability of $\mathbb P(X_n=A\mid X_0=C)$ is obtained by multiplying: \begin{align} \lim_{n\to\infty}\mathbb P(X_n=A\mid X_0=C) &= \lim_{n\to\infty} \mathbb P(X_{\tau+n}=A\mid X_\tau = A)\cdot \mathbb P(X_\tau = A)\\ &= \frac37\cdot\frac13\\ &=\frac17. \end{align}