For a discrete Markov chain with transition probability matrix $P$, we see that the invariant distribution satisfies
$\pi = \pi P^n\tag{1}$
now suppose I can calculate $\Pi:= \lim_n P^n$ analytically. I know from the theory that $\pi_i$ represents the long-run portion spent in the state $i$.
In the concrete example we have the Ehrenfest two urn model with 3 balls. I was able to calculate the transition probabilities as well as $\Pi$. The value $X_n$ of the Markov Chain denotes the number of balls in urn $1$ at time step $n$. What I want to calculate is the long-run portion of time that the first urn was empty, i.e. $\pi_0$. Now its mentioned that one can use $\Pi$ to calculate $\pi_0$ but I'm not quite sure how. Usually, I would just solve equation $(1)$ above. Any clarification would be appreciated
The transition matrix in this case is given as
$$ \begin{equation*}
P =
\begin{pmatrix}
0 & 1 & 0 & 0 \\
\frac{1}{3} & 0 & \frac{2}{3} & 0\\
0 & \frac{2}{3} & 0 & \frac{1}{3}\\
0 & 0 & 1 & 0\\ \end{pmatrix}
\end{equation*}$$
and $$\Pi = \lim_n P^n = \begin{equation*}
\begin{pmatrix}
0.25 & 0 & 0.75 & 0 \\
0 & 0.75 & 0 & 0.25\\
0.25 & 0 & 0.75 & 0\\
0 & 0.75 & 0 & 0.25\\ \end{pmatrix}
\end{equation*}$$
I know from calculating $(1)$ that $\pi_0=0.125$. But how can I get this via $\Pi$?
Best Answer
The "problem" lies in your matrix $P$. The chain is periodic $(2)$ and thus you are not guaranteed that
$$\lim_n P(X_n=j|X_0=i)=\pi_j \forall i,j $$
You would need to calculate the $\pi_0$ vie $\pi_0=\pi_0 P$ or check that the chain is reversible and use the detailed balanced equation if applicable.