I'm giving my first steps in stochastic processes but I'm having some difficulties. See the following example
Suppose that whether or not it rains today depends on previous weather
conditions through the last two days. Specifically, suppose that if it
has rained for the past two days, then it will rain tomorrow with
probability 0.7; if it rained today but not yesterday, then it will
rain tomorrow with probability 0.5; if it rained yesterday but not
today, then it will rain tomorrow with probability 0.4; if it has not
rained in the past two days, then it will rain tomorrow with
probability 0.2.If we let the state at time n depend only on whether or not it is
raining at time n, then the preceding model is not a Markov chain (why
not?). However, we can transform this model into a Markov chain by
saying that the state at any time is determined by the weather
conditions during both that day and the previous day. In other words,
we can say that the process is instate 0 if it rained both today and yesterday,
state 1 if it rained today but not yesterday,
state 2 if it rained yesterday but not today,
state 3 if it did not rain either yesterday or today.
$$P=\begin{bmatrix}0.7&&0&&03&&0\\0.5&&0&&0.5&&0\\0&&0.4&&0&&0.6\\0&&0.2&&0&&0.8\end{bmatrix}$$
I'm wondering how this matrix is computed,
$$P_{ij}=P(state_i|state_j)$$
I am unable to see clearly how these conditionals are calculated.
Best Answer
Denoting the current day as index $i$ (so that yesterday is $i-1$ and the day before yesterday is $i-2$), the previous state comprises two elements, $State_{n-1}=\{S_{i-1},S_{i-2}\}$, where $S_k$ is the state of the weather for day $k$. Thus the current state is $State_{n}=\{S_{i},S_{i-1}\}$
Lettting $R$ be the occurrence of rain on a particular day, so that $\overline{R}$ is when no rain occurs for that day, there are $4$ possible values of the previous state $State_{n-1}$ $$\{\overline{R},\overline{R}\}, \{\overline{R},R\},\{R,\overline{R}\},\{R,R\}$$ For each of these $4$ states, there are only two possible next states:- $$\begin{align} State_n=\{\overline{R},\overline{R}\}\text { or }State_n=\{R,\overline{R}\}\text{ given }State_{n-1}=\{\overline{R},\overline{R}\} \\ State_n=\{\overline{R},\overline{R}\}\text { or }State_n=\{R,\overline{R}\}\text{ given }State_{n-1}=\{\overline{R},R\} \\ State_n=\{\overline{R},R\}\text { or }State_n=\{R,R\}\text{ given }State_{n-1}=\{R,\overline{R}\} \\ State_n=\{\overline{R},R\}\text { or }State_n=\{R,R\}\text{ given }State_{n-1}=\{R,R\} \end{align}$$ With the information given in the question, and based on the constraints of what the next state is based on the current state, you can figure out the structure of the transition matrix.
For example, the information Specifically, suppose that if it has rained for the past two days, then it will rain tomorrow with probability $0.7$, corresponds to $$P(State_{n+1}=\{\overline{R},\overline{R}\}|State_{n}=\{\overline{R},\overline{R}\})=0.7\\\Rightarrow P(State_{n+1}=\{R,\overline{R}\}|State_{n}=\{\overline{R},\overline{R}\})=1-0.7=0.3$$ and if it rained today but not yesterday, then it will rain tomorrow with probability 0.5 corresponds to $$P(State_{n+1}=\{\overline{R},\overline{R}\}|State_{n}=\{\overline{R},R\})=0.5\\\Rightarrow P(State_{n+1}=\{R,\overline{R}\}|State_{n}=\{\overline{R},R\})=1-0.5=0.5$$