We consider a Markov chain on $\{0,1,..\}$ such that the only non zero transitions from state $i$ are $i-1$ and $i+1$ with probabilities respectively $p_i$ and $q_i$.
So we define $\displaystyle \alpha_k=\frac{q_1}{p_1}\frac{q_2}{p_2}…\frac{q_{k-1}}{p_{k-1}}$
I am asked to prove that the chain is transient if and only if
$\sum_{k=1}^{\infty} \alpha_k < \infty$
EDIT: Maybe I found a better way:
As it is a sum of positive terms it can either converge or go to infinity, it can't oscillate.
Assume the sum diverges
Then we have that, if $h_i$ is the probability of reaching $0$ from $i$,
$h_i=1-(1-h_1)\sum_{k=1}^{i}\alpha_k$
But then as all the $h_i\in [0,1]$ we must have $h_1=1$ as otherwise the $h_i$ would be eventually outside the right range.
But then $h_i=1 \forall i$ which means that $0$ is recurrent.
So the chain is recurrent and we proved one direction by contrapositive.
For the converse I did the following:
if the chain is transient then $0$ is transient and then $\forall i$ $(1-h_1)\sum_{k=1}^{i} \alpha_k$ is in $(0,1]$ so it is an increasing bounded sequence and so it converges.
I am not entirely sure that this makes sense though, I think I am missing something…
Any help would be appreciated!
Best Answer
First, I think there is a mistake in your statement: $P(i,i-1)=p_i,\ P(i,i+1)=q_i$ If you $p_i=1,\ q_i=0$, then $0$ must be recurrent, but $\sum_{k=1}^{\infty}\alpha_k=0<\infty$. Hence, $P(i,i+1)=p_i,\ P(i,i-1)=q_i$.
Proof:
Define $\varphi(n):=\sum_{m=1}^{n}\alpha_m=\sum_{m=1}^{n}\prod_{j=1}^{m-1}\frac{q_j}{p_j}$, then $\varphi(X_n)$ is a martingale, and you can check this by definition. If $a<x<b,\ (a,b,x\mbox{ are integers}),\ T=T_a\wedge T_b$, then $\varphi(X_{n\wedge T})$ is a bounded martingale.
We can compute $P_x(T_a>T_b)=\frac{\varphi(x)-\varphi(a)}{\varphi(b)-\varphi(a)}$, let $a=0,b=M$, we obtain $P_x(T_0>T_m)=\frac{\varphi(x)}{\varphi(M)}$ .Then
$$ 0 \mbox{ is recurrent if and only if } \varphi(M)\to \infty \ as\ M\to \infty $$
Note that $\varphi(\infty)=\sum_{k=1}^{\infty} \alpha_k$