I have a Markov chain with state space $E = \{1,2,3,4,5\}$ and transition matrix below:
$$
\begin{bmatrix}
1/2 & 0 & 1/2 & 0 & 0 \\
1/3 & 2/3 & 0 & 0 & 0 \\
0 & 1/4 & 1/4 & 1/4 & 1/4 \\
0 & 0 & 0 & 3/4 & 1/4 \\
0 & 0 & 0 & 1/5 & 4/5
\end{bmatrix}
$$
Given the initial distribution $\pi = (1/2,0,0,1/2,0)$, how would I calculate $\mathbb{P}(X_2 = 4)$? Do I have to multiply $\pi$ by the transition matrix etc.?
Best Answer
If "initial distribution" means the distribution of $X_1$, then the distribution of $X_2$ is $\pi M$, where $M$ is the transition matrix above.
If "initial distribution" means the distribution of $X_0$, then the distribution of $X_2$ is $\pi M^2$.