Consider the following Markov chain, which is as a random walk with reflection at state zero. Classify the states of the chain (positive/null recurrent or transient).
Solution: if $p<\frac{1}{2}$, all the states are positive recurrent; if $p=\frac{1}{2}$, all states are null recurrent, and in the case $p>\frac{1}{2}$ we have that every state is transient.
As the chain is irreducible all the states will be of the same type. Studying the state $0$ seems to be the best option.
I tried proving if $\sum_{n\geq1}p_{00}(n)=\infty$ to see if $0$ is recurrent. $\sum_{n\geq1}p_{00}(2n)=\sum_{n\geq1}{2n\choose n}p^nq^n$$\sim \sum_{n\geq1}\frac{(4pq)^n}{\sqrt{\pi n}}$ (by Stirling), which diverges iff $p=\frac{1}{2}$, but for $p<\frac{1}{2}$ the states are also recurrent…
To prove positive/null recurrence we know that a recurrent state is null-recurrent iff $p_{ii}(n)\to0$ as $n\to\infty$, but $\lim_{n\to\infty}\frac{(4pq)^n}{\sqrt{\pi n}}=0$, and we already now that for $p<\frac{1}{2}$ this should not happen.
EDIT 1
While proving $\sum_{n\geq1}p_{00}(n)=\infty$ I was assuming that $p_{00}(2n+1)=0$, which is clearly wrong as we can go from $0$ to $0$ in an odd number of steps thanks to $p_{00}(1)=q$. Now I'm trying to calculate these $p_{00}(n)$ but I don't succeed.
EDIT 2 – New attempt
Let $T_i$ be the return time of a certain state (i.e. first time we hit $i$ since we leave it). Then the state $i$ is transient if $Pr(T_i<\infty)=1$ and recurrent if $Pr(T_i<\infty)<1$. In this case we have that $Pr(T_i=2n+1)=0$ for $n\geq1$, $Pr(T_i=1)=q,\ Pr(T_i=2n+2)=pq\cdot C_{2n}\cdot(pq)^n\sim$$\sim\frac{pq}{\sqrt\pi}\frac{(4pq)^n}{(n+1)\sqrt n}$ (by Stirling), as we first must go from $0$ to $1$ (and the last step from $1$ to $0$) and then in $1$ we can go right a number of steps always $\geq$ than the ones we go left.
Then, $Pr(T_i<\infty)=Pr(T_i=1)+\sum_{n\geq0}Pr(T_i=2n+2)=q+pq+\frac{pq}{\sqrt\pi}\sum_{n\geq1}\frac{(4pq)^n}{(n+1)\sqrt n}$, but I don't know how to compute this sum.
Best Answer
Have you met the method of proving that the states are positive recurrent by showing that there is a stationary distribution?
For $p<0.5$ the stationary distribution for your problem can be found by solving simple equations like $$\pi_0=\pi_0q+\pi_1q,\pi_1=\pi_0p+\pi_2q, ... $$
A solution is $\pi_0=\frac{1-2p}{1-p},\pi_n=\pi_0(\frac{p}{1-p})^n$ and so all the states are positive recurrent.
Conversely, there is no solution with the $\pi_i$s summing to $1$ if $p\ge 0.5$ and so the states are either null recurrent or transient in that case.