I have the following transition matrix for my Markov Chain:
$$
P= \begin{pmatrix}
1/2 & 1/2 & 0&0&0& \cdots \\
2/3 & 0 & 1/3&0&0&\cdots \\
3/4 & 0& 0&1/4&0&\cdots \\
4/5 & 0&0&0&1/5&\cdots\\
5/6&0&0&0&0&\cdots\\
\vdots&\vdots&\vdots&\vdots&\vdots&\ddots
\end{pmatrix}
$$
defined by
$$ P_{ij} =
\begin{cases}
\frac{i}{i+1}, & \text{if j=1} \\
\frac{1}{i+1}, & \text{j=i+1}\\
0, & \text{otherwise.}
\end{cases}
$$
I've tried finding a pattern for just a 5×5 version of this but I can't seem to find anything unique. In fact, all of these zeroes seem to take on values at higher powers of the matrix.
The only pattern I see is that each element in the 1st column becomes the sum of 1/2 the element in the column plus 1/i+1. They seem to approach .582 for all elements… But how would I show this in the infinite case? I can't envision how I would multiply by the stationary distribution $\pi$.
Can anyone give me some insight on where I should go from here? Thank you greatly in advance!
EDIT: Forgot the actual question. Ugh I'm tired :/
(a) Does the chain have a stationary distribution? If yes, exhibit the distribution. If no, explain why.
(b) Classify the states of the chain.
Best Answer
There is a technique to reduce the number of equations to solve by one. Note that if $x$ satisfies $xP=x$ then (c$x$)$P$ = c$x$, $\forall$c.
Let c = $\frac{1}{\sum_j x_j}$
Then we can have $x$ = $(1, x_1, x_2, ...)$
and $\pi = \frac{1}{1+x_2+x_3+...+x_4} (1,x_2,x_3,...,x_k)$.
Actually solving:
We know the pattern of P which is given in my question. When you multiply,
$\pi P = \pi$
You get the system of equations;
$\frac{1}{2}\pi_1 + \frac{2}{3}\pi_2 + \frac{3}{4}\pi_3+... = \pi_1$
$\frac{1}{2} \pi_1 = \pi_2$
$\frac{1}{3}\pi_2 = \pi_3$
$\frac{1}{4}\pi_3 = \pi_4$
Continue in this manner indefinitely. The pattern is clear, though.
Now, as explained above, set $\pi_1 = 1$.
Then we get $\pi_1 = 1$, $\pi_2 = \frac{1}{2}$, $\pi_3=\frac{1}{6}$, $\pi_4 = \frac{1}{24}$,$...$
Which you can express as factorials.
$\pi_1 = \frac{1}{1!}$, $\pi_2 = \frac{1}{2!}$, $\pi_3=\frac{1}{3!}$, $\pi_4 = \frac{1}{4!}$,$...$
This is actually a Maclaurin Series!
$\sum_{j=0}^{\infty} \frac{1}{j!} = e$
and in our case we start with $j=1$ so we have,
$\sum_{j=1}^{\infty} \frac{1}{j!} = e-1$
So we have,
$\pi = \frac{1}{e-1} (1, \frac{1}{2},\frac{1}{6},...,\frac{1}{j!})$