I was practicing some questions on transition probability matrices and I came up with this question.
You have 3 coins: A (Heads probability 0.2),B (Heads probability 0.4), C (Heads probability 0.6).Plan is to toss one of the 3 coins each minute. Start by tossing A. Subsequently if you toss Heads you coin A next minute. If you toss Tails, you chose at random with equal probability one of coins B or C for your next toss.
$X_n: $type of coin tossed on the nth minute.
Then the one steep transition probability matrix is:
$$
\begin{matrix}
0.2 & 0.4 & 0.4 \\
0.4 & 0.3 & 0.3 \\
0.6 & 0.2 & 0.2 \\
\end{matrix}
$$
a) Determine the asymptotic proportion of minutes you toss heads.
b) how could you compute the probability that you toss Heads on tosses 3 and 10.
My answer for a) is
$\pi_A *0.2 +\pi_B *0.4 +\pi_c* 0.6$ where $\pi_A,\pi_B,\pi_C$ are stationary distribution.
But the given answer is $\pi_A$.Can some one please tell me what is wrong with the way I did it.
For b) Is it P(tossing coin A on 4th toss and tossing coin on 11 th toss)=$P_{AA}^4*P_{AA}
^{11}$ ?
Best Answer
Determining the stable (long term) state probabilities
The solutions of the following system of equations are the stationary state probabilities:
$$\begin{matrix}[\pi_A\ \pi_B \ \pi_C]\\ \\ \\ \end{matrix} \begin{bmatrix} 0.2 & 0.4 & 0.4 \\ 0.4 & 0.3 & 0.3 \\ 0.6 & 0.2 & 0.2 \\ \end{bmatrix} \begin{matrix}=[\pi_A\ \pi_B \ \pi_C]\\ \\ \\\end{matrix} $$ and $$ \pi_A+\pi_B+\pi_C=1 .$$
Note
Here $\pi_A\not=1$. $\pi_A$ is a hypothetical quantity an intuitive definition of which (together with that of $\pi_B$,$\pi_C$) is as follows: "If we started the system choosing a state randomly with probabilities $\pi_A,\pi_B, \pi_C$ then the probability that the next state would be $A$, $B$, or $C$ would be $\pi_A,\ \pi_B, \pi_C$, respectively." Such stationary probabilities do not necessarily exists. You can read about the details here.
Continued
Having performed the vector-matrix multiplication we arrive at a redundant system of equations. After omitting the redundant equation and after some rearrangement we have
$$\begin{matrix} -0.8\pi_A & +& 0.4\pi_B & +& 0.6\pi_C &=0 \\ \ \ 0.4\pi_A & -& 0.7\pi_B& +& 0.2\pi_C&=0 \\ \pi_A&+&\pi_B&+& \pi_C&=1& \end{matrix}$$
The solution, with the help of Alpha, is $$\pi_A=\frac{5}{13}, \ \pi_B=\frac{4}{13},\ \pi_C=\frac{4}{13} .$$
Question a.
We can toss heads in each states, so $$P(\text{heads})=P(\text{heads}|\text{in state} A)\pi_A+P(\text{heads}|\text{in state}B)\pi_B+P(\text{heads}|\text{in state}C)\pi_C=\frac{2}{10}\frac{5}{13}+\frac{4}{10}\frac{4}{13}+\frac{6}{10}\frac{4}{13}=\frac{5}{13}=\pi_A.$$ That is, the result of the tossing is a head in $\frac{5}{13}100$% of the time on the long run.
Remark
It seemed to be accidental that
$$P(\text{heads})=\pi_A.$$
I wondered, if it was? So I took $p_A, p_B, \text{ and }p_C$ as the probabilities of the heads in the states $A, B, \text{ and } C$, respectively.
In order to get the stationary probabilities we have to solve the following system of equations:
$$\begin{matrix} p_A\pi_A & +& p_B\pi_B & +& p_C\pi_C &=\pi_A \\ \frac{1-p_A}{2}\pi_A & +& \frac{1-p_B}{2}\pi_B& +& \frac{1-p_C}{2}\pi_C&=\pi_B \\ \pi_A&+&\pi_B&+& \pi_C&=1& \end{matrix}.$$ The solutions are
$$ \begin{matrix} \pi_A=\frac{p_B+p_C}{-2p_A+p_B+p_C+2}\\ \pi_B=\frac{1-p_A}{-2p_A+p_B+p_C+2}\\ \pi_C=\frac{1-p_A}{-2p_A+p_B+p_C+2} \end{matrix}. $$ At this point, note that $p_A<1$. If $p_A$ was $1$ then the Markov chain would never get out of state $A$. Then, of course, $$1=P(\text{heads})=\pi_A.$$
If $p_A<1$ then the solutions above are valid and we may compute the probability of the heads in general. Watch this:
$$P(\text{heads})=\frac{(p_B+p_C)p_A+(1-p_A)p_B+(1-p_A)p_C}{-2p_A+p_B+p_C+2}=\pi_A \ !$$
Perhaps this is what the OOOOOP wanted to see.
Question b.
We have to go back to the initial state which is $A$. Now, the following vector-matrix product will give the probabilities that we end up in states $A$,$B$, or $C$ at the $\text{n}^{th}$ minute:
$$\begin{matrix}[1\ 0 \ 0]\\ \\ \\\end{matrix} \begin{bmatrix} 0.2 & 0.4 & 0.4 \\ 0.4 & 0.3 & 0.3 \\ 0.6 & 0.2 & 0.2 \\ \end{bmatrix}^n\begin{matrix}=[\pi_A^n\ \pi_B^n \ \pi_C^n]\\ \\ \\\end{matrix} $$
Finally, we have to repeat the steps performed during solution a. But this time we work with the results of the vector-matrix product above with $n=3$ and with $n=10$.