I have a stationary and asymptotic markov chain $\ {X; t = 0,1,…}$ with state space $ Sx = { 1,2,3 } $ and transition matrix P \begin{bmatrix}0.2&0.5&0.3\\0.4&0.3&0.3\\1&0&0\end{bmatrix}with initial distribution $\overline p$ \begin{bmatrix}1/3\\1/3\\1/3\end{bmatrix}
I now want to compute $\ P(X_{3} = 3, X_{2} = 1|X_{1} = 2, X_{0} = 2)$.
The answer is $ P_{2,1} P_{1,3}$ = 0.12.
I have tried several approaches using conditional probability theorems but never seem to get the right answer.
I tried rearranging it to $ \frac{ P(X_{3}=3, X_{2}=1) P(X_{1}=2, X_{0}=2 | X_{3}=3, X_{2}=1)}{P(X_{1}=2, X_{0}=2)} $ and get $ P_{1,3} $
I would appreciate any help here please.
Best Answer
First of all, what you need to do is recall the Markov property. The Markov property tells you the following for any natural number $n$, and $k , x_1,...,x_n$ in the state space: $$ P(X_{n+1} = k | X_1 = x_1,...,X_{n-1} = x_{n-1}, X_n = x_n) = P(X_{n+1} = k | X_n = x_n) $$
That is, only "the present" determines the future : the future is independent of the past at each point.
To make use of this, since you have multiple $X_i$ on the left side of the $|$, you must break it up using the fact $$P(A \cap B | C) = \frac{P(A \cap B \cap C)}{P(C)} = \frac{P(A \cap B \cap C)}{P(B \cap C)}\frac{P(B \cap C)}{P(C)} = P(A | B,C)P(B|C)$$
(Note that $B,C$ is the same as $B \cap C$, it is just that it is notationally easier on the right side of the $|$ to write the comma)
So we start with $K = P(X_3 = 3 ,X_2= 1| X_1 = 2, X_0 = 2)$. Let us do this break up for $A \to X_3 = 3$ and $B \to X_2 = 1$ and $C \to X_1 = 2 , X_0 = 2$ : $$ K = P(A\cap B| C) = P(A | B,C)P(B|C) \\ = \color{green}{P(X_3 = 3 | X_2 = 1 , X_1 = 2 , X_0 = 2)}\color{blue}{P(X_2 = 1 | X_1 = 2 , X_0 = 2)} $$
Now, use the Markov property on the green expression.You get $\color{green}{P(X_3 = 3 | X_2 = 1)} = 0.3$ since this is the transition probability from state $1$ to state $3$.
Similarly ,the blue expression is $\color{blue}{P(X_2 = 1 | X_1 = 2)} = 0.4$ since this is the transition probability from state $2$ to state $1$.
Their product is $0.12$, as desired.