A Markov Chain $(X_n)_n$ has the following transition matrix:
$$P = \begin{bmatrix}
0.1 & 0.3 & 0.6\\
0 & 0.4 & 0.6\\
0.3&0.2&0.5
\end{bmatrix}$$
with initial distribution $\alpha = (0.2, 0.3, 0.5)$.Find the following:
(a) $P(X_7 = 3|X_6 = 2)$
(b) $P(X_9 = 2|X_1 = 2, X5 = 1, X_7 = 3)$
(c) $P(X_0 = 3|X_1 = 1)$
What I understand according to the Example-$7.16$ of book by Symour Lipchuz (Page-$132$) that, $P^n$ is used to find the probabilities of change of states in exactly $n$ steps, and $p^{(m)} \cdot P^n$ is used to find the probability distribution of various states after $m$ steps.
According to my understanding, there are no uses of Initial Distribution in these three computations. Because, multiplication of a $1X3$ and $3X3$ matrices will give a $1X3$ matrix. In that case, the transition matrix won't make sense for three states.
So, my attempted solution is the following:
(a) This question actually asks:
What is the probability that the system changes from state-$2$ to state-$3$ in exactly $1$ step?
So, the answer would be $p(2, 3) = 0.6$
(b) This question actually asks:
What is the probability that the system changes from state-$3$ to state-$2$ in exactly $2$ steps?
$$P^2 = P \cdot P = \begin{bmatrix}
0.1 & 0.3 & 0.6\\
0 & 0.4 & 0.6\\
0.3&0.2&0.5
\end{bmatrix} \cdot \begin{bmatrix}
0.1 & 0.3 & 0.6\\
0 & 0.4 & 0.6\\
0.3&0.2&0.5
\end{bmatrix} = \begin{bmatrix}
0.19&0.27&0.54\\
0.18&0.28&0.54\\
0.18&0.27&0.55
\end{bmatrix} $$
So, the answer would be: $p(3, 2) = 0.27$
(c) I think, this question asks:
What is the probability that the system changes from state-$1$ to state-$3$ in exactly $9$ steps?
Is it? If yes, then it would be $p(1,3)$ from $P^9$.
Am I correct?
Edit:
$P(X_0 = 3|X_1 = 1) = \frac{P(X_0 = 3, X_1 = 1)}{P(X_1 = 1)}$
$\Rightarrow P(X_0 = 3|X_1 = 1) = \frac{P(X_1 = 1, X_0 = 3)}{P(X_1 = 1)}$
$\Rightarrow P(X_0 = 3|X_1 = 1) = \frac{P(X_1 = 1| X_0 = 3)\cdot P(X_0=3)}{P(X_1 = 1)}$
Now,
- from $P$, we have $P(X_1=1|X_0=3) = 0.30$
- from $\alpha$, we have $P(X_0=3) = 0.50$, and
- from $\alpha P$, we have $P(X_1=1) = 0.17$
So,
$P(X_0 = 3|X_1 = 1) = \frac{0.30 \cdot 0.50}{0.17} \approx 0.88$.
Is this a correct calculation?
Best Answer
You have part (a) and part (b) down. For part (c), we need that initial distribution.
What is the probability that the system is in state 1 after one step?
What is the probability that the system starts in state 3 and is then in state 1 after 1 step?
Can you use these to find the conditional probability this part asks for?
And no, there's no cyclic behavior here. The sequence of $X_n$ keeps going for all positive integers $n$, and it's not periodic.
[In response to the added material in the question]
Yes, that's a correct calculation. You have now fully solved the problem.