[Math] How to get the proportion of time of a state of this markov chains process

markov chainsprobability

Suppose each day there is a $0.2$ probability will rain in the morning. $P(\text{rains afternoon}|\text{rain morning})=0.6$ and $P(\text{rains afternoon}|\text{not rain morning})=0.3$. Suppose John would go to the office in the morning and leave in the afternoon and he totally get 3 umbrellas. Let $X_n$ be the number of umbrellas storing in his office at $nth$ night. Whats the proportion of time he has $0$ umbrella in his office at night?

I tried to make the transition probability matrix and find the stationary distribution at state $0$ and found that $\pi_0=0.19$ but seems not consistent with the answer but i don't know what is the mistake. Any hints or solution are welcome.

$P_{transition} =\begin{pmatrix}0.92&0.08&0&0\\0.24&0.68&0.08&0\\0&0.24&0.68&0.08\\0&0&0.24&0.76\end{pmatrix}$
so $P-I=\begin{pmatrix}-0.08&0.08&0&0\\0.24&-0.32&0.08&0\\0&0.24&-0.32&0.08\\0&0&0.24&-0.24\end{pmatrix}$

Best Answer

It would help a lot if you gave some indication as to how you got $0.19$ --- then maybe someone would see an arithmetical mistake, and we could all go home. Since you haven't done that, and since I'm not inclined to do the whole problem, I'll just indicate how one might get proceed.

$X_n$ goes up by one when it rains in the morning and not in the afternoon. The probability of this is $.08$ (unless $X_n=3$, in which case the probability is zero).

$X_n$ goes down by one when it doesn't rain in the morning but does rain in the afternoon. The probability of this is $.24$ (unless $X_n=0$, in which case the probability is zero).

This gives you all the transition probabilities; you can now figure out the eigenvector for the eigenvalue $1$ and thus the answer to the question.