I am not understanding how is the transition probability matrix of the following example constructed.
Suppose that whether or not it rains today depends on previous weather conditions through the last two days. Specifically, suppose that if it has rained for the past two days, then it will rain tomorrow with probability $0.7$; if it rained today but not yesterday, then it will rain tomorrow with probability $0.5$; if it rained yesterday but not today, then it will rain tomorrow with probability $0.4$; if it has not rained in the past two days, then it will rain tomorrow with probability $0.2$.
Let
state $0$ if it rained both today and yesterday,
state $1$ if it rained today but not yesterday,
state $2$ if it rained yesterday but not today,
state $3$ if it did not rain either yesterday or today.
The preceding would then represent a four-state Markov chain having a transition
probability matrix
$$P=
\begin{bmatrix}
0.7 & 0 & 0.3 & 0 \\
0.5 & 0 & 0.5 & 0 \\
0 & 0.4 & 0 & 0.6 \\
0 & 0.2 & 0 & 0.8 \\
\end{bmatrix}.
$$
Why is $P_{10}=0.5$ ? As the 2nd row corresponding to state $1$ represents it rained today but not yesterday, can't i assign the $0.5$ in $P_{11}$ or in $P_{13}$ ?
$\bullet$ Second portion of the example is : Given that it rained on Monday and
Tuesday, what is the probability that it will rain on Thursday?
For solving this we have to compute two-step transition matrix. But if I were asked to find the probability that it will rain on Friday, would I have to compute three-step transition matrix ? Will I count the step from Tuesday?
And the book says that rain on Thursday is equivalent to the process being in either state $0$ or state $1$. Why it is not other states rather than state $0$ or state $1$?
Best Answer
First part:
Let $a,b,c$ represent $3$ consecutive days. Since we are in state $1$, that means we have the sequence $(a,b) = \text{(no rain, rain)}$. In order to jump onto state $0$, there must hold $(b,c) = \text{(rain, rain)}$. Then we have the sequence $(a,b,c) = \text{(no rain, rain, rain)}$. According to the assumptions, starting from $(a,b)$ we can reach $c$ with probability $p=0.5$.
Also, $P_{11} = 0$. Why? If we still have $3$ consecutive days $a,b,c$ then it must hold $(a,b) = \text{(no rain, rain)}$ and $(b,c) = \text{(no rain, rain)}$, which can't happen.
Second part:
Notice that we start from state $0$, thus $\pi(0) = \begin{bmatrix} 1& 0 & 0 & 0\end{bmatrix}$ and we are going to evaluate the probability: $$\pi(0)\cdot P^2 = \begin{bmatrix} 0.49 & 0.12 & 0.21 & 0.18\end{bmatrix}. $$ Thus, the probability that it rains on Thursday is going to be $p=0.49+ 0.12 = 0.61$ (see part $3$).
Third part:
From part $2$ it is known that the initial state is the state $1$. Assuming that we have the sequence $(a,b,c,d)$ with $a$ corresponding to the first day (Monday) and $d$ correspond to the last day (Thursday). Thus, we want the following to hold: $$(a,b,c,d) = \text{(rain, rain, x, rain)}.$$ $x$ could either represent a rainy day or a non - rainy day. Thus, the are $2$ paths.
1: $(a,b,c,d) = \text{(rain, rain, rain, rain)}$
2: $(a,b,c,d) = \text{(rain, rain, no rain, rain)}$
Τhus, $(c,d)$ is going to be either (rain, rain), which indeed corresponds to state $0$ or (no rain, rain), which corresponds to state $1$.
Speaking with term of states the first $4-tuple$ corresponds to the path $0\to 0\to 0$, thus we have $p_{00}\cdot p_{00}= 0.7^2=0.49$ and the second $4-tuple$ corresponds to the path $0\to 2\to 1$, thus $p_{02}\cdot p_{21} = 0.3 \cdot 0.4 = 0.12$. Adding the two probabilities, leads us to the answer of the second part.