[Math] Markov chains – transition matrix – probabilty formula and application help

markov chainsstochastic-processes

Consider the $3\times3$ transition matrix

$$
\begin{array}{c|ccc}
&A &B &C \\
\hline
A &0.2 & 0.3 &0.5\\
B &0.3 & 0.5 &0.2\\
C &0.3 & 0.3 & 0.4
\end{array}.
$$
Given that the chain starts in state $B$ what is the probability that after three transitions it is back in state $B$ having visited $A$ and $C$?

I have a few exam paper questions similar to this but I can only find one sort of formula related to this in my notes and it's not very clear and no examples. I was wondering if someone could give me a formula for these types of problems and show me how to apply it to this question and how to use it for others?

Many thanks in advance. 🙂

Best Answer

$$B\Rightarrow C\Rightarrow A\Rightarrow B$$ $$P(x_0=B,x_1=C,x_2=A,x_3=B)=P(x_0=B)\ P(x_1=C|x_0=B)$$ $$\qquad \qquad \qquad P(x_2=A|x_1=C)\ P(x_3=B|x_2=A)$$ $$P(x_0=B,x_1=C,x_2=A,x_3=B)=1\times 0.2\times 0.3\times 0.3=0.018$$ or $$B\Rightarrow A\Rightarrow C\Rightarrow B$$ $$P(x_0=B,x_1=A,x_2=C,x_3=B)=P(x_0=B)\ P(x_1=A|x_0=B)$$ $$\qquad \qquad \qquad P(x_2=C|x_1=A)\ P(x_3=B|x_2=C)$$ $$P(x_0=B,x_1=A,x_2=C,x_3=B)=1\times 0.3\times 0.5\times 0.3=0.045$$ and in total $$P=0.063$$ -----------Transition Matrix------------ \begin{array}{c|ccc} &A &B &C \\ \hline A &P(A|A) &P(B|A) &P(C|A)\\ B &P(A|B) &P(B|B) &P(C|B)\\ C &P(A|C) &P(B|C) &P(C|C) \end{array} which means if you select (2,1) it means that you look up for the probability of transition from B to A. Rows are for current state and columns for next state.

Related Question