Markov chain problem: probability of raining at any given day

markov chainsprobability

I have a very annoying problem. I am just beginning to learn the Markov chains, and I cant figure out how should I solve this problem.

So I have a transition table showing the state transitions of rain and sunny for yesterday and today.


———————-| today sunny | today raining|

yesterday sunny—| 0.9———–| 0.1———-|

yesterday raining-| 0.5————| 0.5———-|


I do not know anything about the actual state and I should find the probability of raining at any given day (so kind of probability of training after an infinite number of days).

How should I calculate this? (My problem that is not a finite problem and I am not aware of the actual status).

Thank you for your help in advance!

Best Answer

Let $x=(x_1,x_2)$ be a steady state, where $x_1$ and $x_2$ are the probabilities of a sunny and a rainy day, respectively. Then: $$x_1=x_0M;\\ x_2=x_1M;\\ \vdots\\ x_{n+1}=x_nM \Rightarrow \\ x=xM \Rightarrow \\ \begin{pmatrix}x_1&x_2\end{pmatrix}=\begin{pmatrix}x_1&x_2\end{pmatrix}\begin{pmatrix}0.9&0.1\\ 0.5&0.5\end{pmatrix} \Rightarrow \\ \begin{cases}0.1x_1-0.5x_2=0 \\ x_1+x_2=1\end{cases} \Rightarrow \\ (x_1,x_2)=(\frac56,\frac16)$$ Hence, the probability of a rainy day is $\frac16$.