Let ${X_n : n = 0, 1, 2, . . .}$ denote a Markov chain with the states $S = {1, 2, 3}$ and transition matrix P given by
$$
\begin{bmatrix}
0 & 0.5 & 0.5 \\
0.1 & 0 & 0.9 \\
0.8 & 0.2 & 0
\end{bmatrix}
$$
Determine whether the Markov chain has a marginal distribution. Determine the it and explain how you found it. Also determine whether the boundary distribution is unique.
So i know that an irreducible and aperiodic markov chain has a unique solution, which also is the limiting distribution. In this case, the stationary distribution is a limiting distribution. To find the stationary distribution i need to solve the following:
\begin{align*}
\pi_1 &= \frac{1}{2}\pi_2 + \frac{1}{2}\pi_3 \\
\pi_2 &= \frac{1}{10}\pi_1 + \frac{9}{10}\pi_3 \\
\pi_3 &= \frac{8}{10}\pi_1 + \frac{2}{10}\pi_2 \\
1 &= \pi_1 + \pi_2 + \pi_3
\end{align*}
I have solved it and got that $\pi_1 = \frac{1}{3}$, $\pi_2 = \frac{1}{3}$ and $\pi_3 = \frac{1}{3}$
Now that i have found my stationary distribution, which is also the limiting distribution what do i do with it? Is my limiting distribution the marginal distribution?
I know that the desired marginal distribution that i need to find is $(0.35, 0.25, 0.4)$ but i can't manage to get this.
Best Answer
I have most of my information from the book Introduction to Probability, Statistics, and Random Processes by Hossein Pishro-Nik and will reference the pages i have looked at for the PDF version here and there!
We have that an irreducible and aperiodic Markov chain has a unique solution, which is also a limiting distribution of the Markov chain. In this case, if the chain is also aperiodic, we conclude that the stationary distribution is a limiting distribution (page 871).
By using RStudio you get the Markov chain is both irreducible and aperiodic and therefore has a unique solution, which is the stationary distribution, which is then a limiting distribution. The stationary distribution can be found by solving $\pi = \pi P$ which is just the following equations (Example 11.15 on page 872): \begin{align*} \pi_1 &= \frac{1}{10}\pi_2 + \frac{8}{10}\pi_3 \\ \pi_2 &= \frac{1}{2}\pi_1 + \frac{2}{10}\pi_3 \\ \pi_3 &= \frac{1}{2}\pi_1 + \frac{9}{10}\pi_2 \\ 1 &= \pi_1 + \pi_2 + \pi_3 \end{align*} After which you get the stationary distribution, which is also the limit distribution: $\pi_1 = 0.35$, $\pi_2 = 0.25$ og $\pi_1 = 0.40$.