[Math] Forming the transition matrix for Markov chain, given a word description of transition probabilities

markov chainsprobability

I have just started learning about Markov chain and have a trouble determining appropriate transition matrix:

Suppose that whether or not it rains today depends on previous weather conditions through the last three days. Show how this system may be analyzed by using a Markov chain by determining how many states are needed and describing the general form of the P matrix. Also suppose that if it has rained for the past three days, then it well rain today with probability .8; if it did not rain for any of the past three days, then it will rain today with probability.2; and if any other case, the weather today will, with probability .6., be the same as yesterday. Determine the matrix P for this Markov chain.

This question has a very detailed explanation but I still don't get it.

Can someone please explain me at least one row? Also how are you suppose to read notations, does YYYY= P(00) means that it will rain today given that it has rained for the past three days? For instance, P(00) = Y YYY = 0.8 because it has rained for the past three days (YYY) then it will rain today.

P(10) = Y NYY = 0. Why 0? Why it isnt 0.6? but P(30) = Y YYN =0.6
I have spent so much time looking for explanations but every website just gives transition matrix without explaining me obtained values.

Best Answer

Suppose it is Friday (Fr), and you are asked to determine the probability that it will rain tomorrow on Saturday (Sa). You will need to know whether it rained yesterday, Thursday (Th), and two days ago on Wednesday (We). Given the information in the problem, you can make the following table:

$$\begin{array} {c|c} \text{Rained on} & \text{Will Rain on} \\ \begin{array} {ccc} \text{We} & \text{Th} & \text{Fr} \\ \hline Y & Y & Y \\ Y & Y & N \\ Y & N & Y \\ Y & N & N \\ N & Y & Y \\ N & Y & N \\ N & N & Y \\ N & N & N \\ \end{array}& \begin{array} {c} \text{Sa} \\ \hline 0.8 \\ 0.4 \\ 0.6 \\ 0.4 \\ 0.6 \\ 0.4 \\ 0.6 \\ 0.2 \\ \end{array} \end{array}$$

Now this is a transition table , but the output states are not the same as the input states. The input states are the rain of 3 days; the input states are the rain of 1 day. So to make a transition matrix, we need the output states to be the same as the input states: 3 days to 3 days, {We, Th, Fr} to {Th, Fr, Sa}.

$$\begin{array} {c|c} \text{Rained on} & \text{Will Rain on Th Fr Sa } \\ \begin{array} {ccc} \text{We} & \text{Th} & \text{Fr} \\ \hline Y & Y & Y \\ Y & Y & N \\ Y & N & Y \\ Y & N & N \\ N & Y & Y \\ N & Y & N \\ N & N & Y \\ N & N & N \\ \end{array}& \begin{array} {c} \text{YYY} & \text{YYN} & \text{YNY} & \text{YNN} & \text{NYY} & \text{NYN} & \text{NNY} & \text{NNN} \\ \hline 0.8 & 0.2 \\ & & 0.6 & 0.4 \\ & & & & 0.6 & 0.4 \\ & & & & & & 0.6 & 0.4 \\ 0.6 & 0.4 \\ & & 0.6 & 0.4 \\ & & & & 0.6 & 0.4 \\ & & & & & & 0.2 & 0.8 \\ \end{array} \end{array}$$

Each blank entry is an impossible transition. For example, it's impossible to transition from a Friday where it rained 3 days in a row to a Saturday where it was clear 3 days in a row (the top right corner). So your final transition matrix is:

$$\begin{bmatrix} 0.8 & 0.2 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0.4 & 0.6 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0.6 & 0.4 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0.4 & 0.6 \\ 0.6 & 0.4 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0.4 & 0.6 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0.6 & 0.4 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0.2 & 0.8 \\ \end{bmatrix}$$

The "notation" (if it can even be called that) that your guide is using is replacing the yes/no sequence with a number (So YYY is 0, NNN is 7, it is quite confusing). $P_{0,0}$ is the probability that it rained for the past 3 days and tomorrow will also be a day where it has rained for 3 days.

Related Question