Let $\mathcal{X}=(X_n:n\in\mathbb{N}_0)$ denote a Markov chain with state space $E=\{1,\dots,5\}$ and transition matrix
$$P=\pmatrix{1/2&0&1/2&0&0\\1/3&2/3&0&0&0\\0&1/4&1/4&1/4&1/4\\0&0&0&3/4&1/4\\0&0&0&1/5&4/5}$$
Compute the probabilities $\mathbb{P}(X_2=5|X_0=1)$ and $\mathbb{P}(X_3=1|X_0=1)$.
Given an initial distribution $\pi=(1/2,0,0,1/2,0)$, compute $\mathbb{P}(X_2=4)$.
I've got the transient states as $1,2,3$. And the recurrent states as $4,5$, and the communication classes I think are $\{1,2,3\}$ and $\{4,5\}$.
1) To calculate $\mathbb{P}(X_2 = 5|X_0 = 1)$, is it just finding $P^2_{(1,5)}$? Which equals $1/8$?
2) For $\mathbb{P}(X_3 = 1|X_0=1)$, I tried finding $P^3_{(1,1)}$ which I got $1/24$. Is that correct?
3) For finding $\mathbb{P}(X_2=4)$, do I just take $π(P^2)$?
Best Answer
The theoretical formulas you suggest are correct. For sparse transition matrices like the one you consider, a simple method is to determine the paths leading to the events one is interested in.
For example, the event that $X_0=1$ and $X_2=5$ corresponds to the unique path $1\to3\to5$, which, conditionally on $X_0=1$, has probability $P(1,3)P(3,5)=\frac18$.
Likewise, the event that $X_0=1$ and $X_3=1$ corresponds to the two paths $1\to1\to1\to1$ and $1\to3\to2\to1$, which, conditionally on $X_0=1$, have respective probabilities $P(1,1)P(1,1)P(1,1)=\frac18$ and $P(1,3)P(3,2)P(2,1)=\frac1{24}$, hence the result is $\frac18+\frac1{24}=\frac16$.
Finally, to evaluate the probability that $X_2=4$, consider that $X_0=1$ or $X_0=4$ hence the three relevant paths are $1\to3\to4$, $4\to4\to4$ and $4\to5\to4$, with respective probabilities $\frac18$, $\frac9{16}$ and $\frac1{20}$, to be weighted by the probabilities that $X_0=1$ or $X_0=4$, hence the final result is $\frac12(\frac18+\frac9{16}+\frac1{20})=\frac{59}{160}$.