[Math] Relationship between a stationary distribution for a random walk and the hitting time at some position

markov chainsprobabilityrandom walk

In a previous question of mine, I asked for the probability distribution of an agent taking a biased walk on the positive integers (with a reflecting boundary at the origin):

Probability distribution for the position of a biased random walker on the positive integers

Here, for a $+1$ step for the walker has probability $p$, for a $-1$ step (away from the origin) the walker has probability $q$, and since $(p+q) \leq 1$, we allow the walker to stay in place with probability $r = 1 – (p+q)$. In order to prevent this walk from becoming transient, we set $p \leq q$.

By solving the relation:

$P(X=n)= p P(X=n-1) +(1-p-q) P(X=n) + q P(X=n+1)$

The user Henry provided a rather nice solution for the stationary distribution that sums to unity and matches up will with simulations with ~$10^7$ or more steps:

$P(X=n) = \left(\frac{p}{q}\right)^n \left(1-\frac{p}{q}\right)$

My question is as follows: Say we initialize our walking agent at some position $0 \leq i < M$, and we're interested in the average hitting time at position $M$. Say we also calculate some sum:

$S = \sum_{n=M}^{\infty} \left(\frac{p}{q}\right)^n \left(1-\frac{p}{q}\right)$

Considering that the stationary distribution $P(X=n)$ is a smoothly decreasing analytic function, can we use $S$ to say anything about the average hitting time, $H(M)$ for the walker agent at position $M$?

Best Answer

No, you can't say anything about the hitting time from the stationary distribution. Consider a random walk in which the state remains unchanged with probability $1-s$ and behaves according to the present random walk with probability $s$. Then the stationary distribution is the same, but all expected times are divided by $s$ You can make $s$ arbitrarily small and thus make the expected times arbitrarily large without changing the stationary distribution.