[Math] Markov Process and interpretation of the transition matrices

markov chainsprobabilitytransition matrix

Suppose that:
Coin $1$ has probability of $0.7$ of coming up heads
Coin $2$ has probability of $0.6$ of coming up heads

If the coin flipped today comes up:
heads: then we select coin $1$ to flip tomorrow,
tails: then we select coin $2$ to flip tomorrow.

If the coin intially flipped is equally likely to be coin $1$ or coin $2$, then what is the probability that the coin flipped on the third day after the initial flip is coin $1$?
This is what they did:

Let $X_n$ denote the label of the coin that is flipped on the $n$th day after the initial flip. X_n is a Markov chain.
The transition probability matrix is given by
$$P = \begin{pmatrix}0.7 & 0.3 \\ 0.6 & 0.4\end{pmatrix}$$
So
$$P^(2) = P^2 = \begin{pmatrix}0.67 & 0.33 \\ 0.66 & 0.34\end{pmatrix}\\P^(3) = P^3 = \begin{pmatrix} 0.667 & 0.333 \\ 0.666 & 0.334\end{pmatrix}.$$
Hence the required probability is
$$\mathbb{P}(X_3 = 1) \\= \mathbb{P}(X_3 = 1|X_0 = 1)\mathbb{P}(X_0 = 1) + \mathbb{P}(X_3 = 1|X_0 = 2)\mathbb{P}(X_0 = 2)\\ = \frac{1}{2}(P^{3}_{11} + P^{3}_{21}) = 0.6665.$$

I understand how they got the $P$, but I think I'm not interpreting it correctly. How did they go from the first "required probability" line to the first equality? Is this using $\mathbb{P}(X=x|Y=y) = \frac{\mathbb{P}(X=x \& Y=y)}{\mathbb{P}(Y=y)}$? I also don't see why they used those matrix components in the last line.

Best Answer

You're entirely correct about how they went from line 1 to line 2. Since the only possible values of $X_{0}$ are $1,2$, $$\mathbb{P}(X_{3}=1) = \mathbb{P}(X_{3}=1 \cap X_{0}=1) + \mathbb{P}(X_{3}=1 \cap X_{0}=2)$$ and from there, using $\mathbb{P}(A \cap B) \equiv \mathbb{P}(A|B)\mathbb{P}(B)$ the second line is obtained.

As for the last line, the matrix component $P_{ij}$ here represents the probability that we transition from state $i$ to state $j$ after one flip; similarly, $P^{3}_{ij}$ represents the probability from transitioning from state $i$ to state $j$ after 3 flips, which we can also write as $\mathbb{P}(X_{3}=j|X_{0}=i)$.

The whole point of the matrix formulation is so you can represent your probability state as a row vector $x$. In this case, initially $x = \left(\array{\frac{1}{2} &\frac{1}{2}}\right)$ since you are equally likely to start with either coin. By right-multiplying by $P^{n}$, the resulting row vector gives the probabilities of using each coin on the $n$th flip.

Related Question