Solved – Ordering of the elements in a transition matrix (Markov Chain)

markov chain

I am bit confused after solving the following Markov Chain exercise from a textbook

Suppose that a student will be either on time or late for a particular
class, and that the events that he is on time or late for the class on
successive days form a markov Chain with stationary transition
probabilities. Suppose also that if he is late on a given day, then
the probability that he will be on time the next day is 0.8.
Furthermore, if he is on time on a given day, then the probability
that he will be late the next day is 0.5

If the student is late on the first day of class, what is the
probability that he will be on time on the fourth day of class?

I solved the exercise by defining the following transition matrix $P$, where $L$ represents the event that the student is late and $T$ represents the event that the student arrives on time:

$$
\begin{array}{lrr}
& \mbox{T} & \mbox{L} \\
\mbox{L} & 0.8 & 0.2 \\
\mbox{T} & 0.5 & 0.5\end{array}
$$

Then, I can answer just by calculating $P^3$:

$$
\begin{array}{lrr}
& \mbox{T} & \mbox{L} \\
\mbox{L} & 0.772 & 0.278 \\
\mbox{T} & 0.695 & 0.0305\end{array}
$$

Therefore, the answer is 0.722.

However, I have realized that if I arrange the elements of the transition matrix $P$ in a different way:

$$
\begin{array}{lrr}
& \mbox{L} & \mbox{T} \\
\mbox{L} & 0.2 & 0.8 \\
\mbox{T} & 0.5 & 0.5\end{array}
$$

then $P^3$ is different to what I obtained before, thus changing the probability that the student is on time on the fourth day of class after being late on the first day to 0.632. This second transition matrix is the one that can be found in the solutions manual of the textbook.

Assuming that the probability shouldn't change based on the arrangement of the elements in the transition matrix, what is wrong with my first version of P?

Best Answer

The problem is with your first matrix: when you arrange your values like that, $P^3$ does not represent the probabilities you are looking for. To see that, consider $P^2$ and calculate the first (top left) value of the product: it will be equal to $$P(L|T)P(L|T) + P(L|L)P(T|T),$$ which is not what you want.

When you arrange your values correctly, like in the second matrix, everything works like it should. The top left value of $P^2$, for example, is equal to

$$P(L|L)P(L|L) + P(L|T)P(T|L),$$ which is equal to probability of being late on third day, given that you were late on the first.

Related Question