Markov chains are widely used in modeling several natural and social processes. Consider the following three-state Markov chain modeling the daily weather in Boston. Each day can be sunny, partly cloudy, or rainy. The Markov chain gives you the probability that the next day is sunny, partly cloudy, or rainy, given the state of the current day. For instance, if today is sunny, then the probability that tomorrow is sunny is 0.5, the probability that tomorrow is partly cloudy is 0.4, and the probability that tomorrow is rainy is 0.1.
Analyze the above Markov chain to answer the following questions. For the last three parts, you need to solve the system of linear equations that give the stationary distribution of the Markov chain.
If today is rainy, what is the probability that the next two days are also rainy? /100
If today is rainy, what is the probability that the day after tomorrow is sunny? /100
In the stationary distribution, what is the probability of any day being sunny? /100
In the stationary distribution, what is the probability of any day being partly cloudy? /100
In the stationary distribution, what is the probability of any day being rainy? /100
I understand how it works with the probability of the next day, but i'm having trouble with the day after. I'm not sure how you would calculate the probability for the next two days since the chain shows only the next day?
Best Answer
The transition matrix $P$ has entries $p_{ij}$ which are the probability of transitioning from $i$ to $j$. So if your states are listed as "sunny,partly cloudy,rainy" then
$$P=\begin{bmatrix} 0.5 & 0.4 & 0.1 \\ 0.4 & 0.5 & 0.1 \\ 0.2 & 0.2 & 0.6 \end{bmatrix}$$
Given an initial distribution $\lambda$ (written as a row vector), the distribution at time $n$ is $\lambda P^n$. (If you prefer, you can work with a column vector and deal with $(P^T)^n \lambda$.) So you can find your second answer by taking $\lambda=[0,0,1]$ and calculating $\lambda P^2$.
The first answer is slightly different because you are being asked about the probability that it rains two days in a row. Here you'll want to use the independence: the probability of being rainy for the next two days is the probability to go from rainy to rainy, and then rainy to rainy again.