Probability Theory – Hitting Probability of Biased Random Walk on the Integer Line

probabilityprobability theoryrandom walk

Lets say we start at point 1. Each successive point you have a, say, 2/3 chance of increasing your position by 1 and a 1/3 chance of decreasing your position by 1. The walk ends when you reach 0.

The question, what is the probability that you will eventually reach 0?

Also, is there any generalization for different probabilities or different starting positions or different rules (say you increase 2 and decrease 1)?

NOTE: I have never taken a course that considered random walks. So, if possible, could no prior knowledge of random walks be assumed?

Best Answer

One asks for the probability $r$ starting at $1$ to eventually reach $0$. The dynamics is invariant by translations hence $r$ is also the probability starting at $2$ to eventually reach $1$. Consider the first step of a random walk starting at $1$. Either the first step is to $0$ then one hits $0$ eventually since one is already at $0$. Or the first step is to $2$ then to hit $0$, one must first hit $1$ starting from $2$ and after that, hit $0$ starting from $1$. This yields the equation $r=\frac13+\frac23r^2$, whose solutions are $r=1$ and $r=\frac12$.

If $r=1$, let us assume the random walk continues with the same dynamics after its first return to $0$. The new portion of the walk is distributed as before hence one returns to $0$ a second time. And so on, hence, calling $X_n$ the position at time $n$, one sees that $X_n=0$ for infinitely many times $n$.

Consider now the homogeneous random walk $(Y_n)$ on the whole integer line whose steps are $+1$ and $-1$ with probabilities $\frac23$ and $\frac13$ respectively. Then $Y_n=Z_1+\cdots+Z_n$ where $(Z_n)$ is i.i.d. and $\mathrm E(Z_n)=\frac13(-1)+\frac23(+1)=\frac13\gt0$. By the strong law of large numbers, $\frac1nY_n\to\frac13$. One can recover $(X_n)$ from $(Y_n)$ through the change of time $\tau_{n+1}=\min\{k\gt\tau_n\mid Y_k\geqslant0,\,Y_k\ne Y_{\tau_n}\}$ for every $n\geqslant0$, and $\tau_0=0$. Then $X_n=Y_{\tau_n}$ and $\tau_n\geqslant n$ hence $\frac1nX_n=\frac1nY_{\tau_n}\geqslant\frac1{\tau_n}Y_{\tau_n}$. One sees that $\liminf\limits_{n\to\infty}\frac1nX_n\geqslant\lim\limits_{n\to\infty}\frac1nY_n=\frac13$.

This is impossible if $X_n=0$ infinitely often, hence $r=\frac12$.

For a random walk whose steps are $+1$ and $-1$ with probability $p\gt\frac12$ and $1-p$ respectively, the same argument yields $r=\frac{1-p}p$.

For a random walk whose steps are $+2$ and $-1$ with probability $p$ and $1-p$ respectively, the crucial argument that to reach $0$ from $n\gt0$, one must reach $n-1$ from $n$, then reach $n-2$ from $n-1$, and so on, is still valid. Hence $r=pr^3+1-p$. If $r\gt\frac13$ the drift $p(+2)+(1-p)(-1)=3p-1$ is positive and one knows that $r\lt1$ hence $r$ is the positive root of the equation $p(r^2+r)=1-p$, that is, $r=\frac1{2p}\left(\sqrt{4p-3p^2}-p\right)$.

The same trick can be applied to any random walk whose steps are almost surely $\geqslant-1$.

Related Question