A Markov chain $\{X_n,n\geq0\}$ has state space $\{0,1,2,…\}$ and transitions from each $i$ to $i+1$ with probability $1-\frac{1}{2i^\alpha}$ and to $0$ with probability $\frac{1}{2i^\alpha}$. It transitions from $0$ to $1$ with probability $1$. Show that the chain is irreducible. Determine if $0$ is transient or recurrent depending on $\alpha$.
Given two states $i$ and $j$, we can find a path that goes from $i$ and $j$ and one that goes from $j$ to $i$, thus we only have one class and the chain is irreducible.
Now, let $T_0$ be the return time of $0$ (i.e. first time we hit $i$ since we leave it). Then the state $i$ is transient if $Pr(T_i<\infty)=1$ and recurrent if $Pr(T_i<\infty)<1$.
If $f_{00}(n)=Pr(T_0=n)$, we have that $f_{00}(1)=0,\ f_{00}(2)=\frac{1}{2}$, and for $n>2$, $\ f_{00}(n)=\frac{1}{2(n-1)^\alpha}\prod_{i=1}^{n-2}(1-\frac{1}{2i^\alpha})$.
$Pr(T_i<\infty)=\sum_{n\geq1}f_{00}(n)=\frac{1}{2}+\sum_{n\geq3}\frac{1}{2(n-1)^\alpha}\prod_{i=1}^{n-2}(1-\frac{1}{2i^\alpha})$
But I don't know how to compute this sum. Could you help me? Is there an easier way to solve the problem? Thanks in advance!
Best Answer
Equivalently, you can compute $\Pr(T_0=\infty)$, and then the state $0$ will be recurrent if and only if this probability is $0$. Note that by continuity of measure, \begin{equation} \Pr(T_0=\infty)=\lim_{n\to \infty} \Pr(T_0\geq n+1)=\lim_{n\to \infty} \prod_{i=1}^n \left( 1-\frac{1}{2i^{\alpha}}\right). \end{equation} It is a standard result that this sequence converges to a nonzero number if and only if $\sum_{i=1}^{\infty} \frac{1}{2i^{\alpha}}<\infty$. Of course, this occurs if and only if $\alpha>1$ by the usual divergence of the harmonic series. As a sanity check, larger $\alpha$ makes it less likely we return to $0$. Therefore, this random walk will be recurrent if and only if $\alpha\leq 1$.