This problem is from Exercise 5.5.6 in Durrett's Probability: Theory and Examples, 5/E,
Use Theorems 5.5.7 and 5.5.9 to show that for simple random walk on $\mathbb{Z}$, if we start from $k$ the expected number of visits to $k$ before hitting $0$ is $2k.$
Theorem 5.5.7 and 5.5.9 are as follows:
Theorem 5.5.7. Let $x$ be a recurrent state in a Markov chain, and $T=\inf(n\geq 1:X_n=x).$ Then $$\mu_x(y)=E_x\left(\sum_{n=0}^{T-1}1_{X_n=y}\right)=\sum_{n=0}^\infty P_x(X_n=y,T>n) $$ defines a stationary measure.
Theorem 5.5.9. If the transition probability $p$ is irreducible and recurrent (i.e. all states are) then the stationary measure is unique up to constant multiplication.
My attempt: Theorem 5.5.7 and 5.5.9 only apply to the case when the starting point and ending point are the same, but is there any means to extend those theorems to the case when the starting and ending points are distinct?
Any hints or advices are welcome!
Best Answer
One approach - I don't know if it's the intended one - is to replace the simple random walk by another irreducible and recurrent Markov chain. For this problem, let's take the state space to be $\{0,1,2,\dots\}$ where states $\{1,2,\dots\}$ act like the simple random walk on $\mathbb Z$, but state $0$ transitions to state $k$ with probability $1$.
We can check that the following measure is stationary for this random walk: $$\mu(x) = \begin{cases} 1 & x=0, \\ 2x & 1 \le x \le k, \\ 2k & x > k. \end{cases}$$ To find this, start with $\mu(0)=1$. Stationarity at $0$ says $\mu(0) = \frac12\mu(1)$, so $\mu(1)=2$. Stationarity at $1$ says $\mu(1) = \frac12 \mu(2)$, so $\mu(2) = 4$. Stationarity at $2$ says $\mu(2) = \frac12\mu(1) + \frac12\mu(3)$, so $\mu(3) = 6$... from there, we can induct, or just guess the formula and verify it.
On the other hand, consider the Theorem 5.5.7 measure $\mu_0$ for this random walk. It has $\mu_0(0) = 1$, and $\mu_0(k)$ is the number of visits to $k$, starting from $0$, before returning to $0$. By Theorem 5.5.9, since $\mu_0(0) = \mu(0)=1$, $\mu_0(k) = \mu(k) = 2k$.
Because the first step in this random walk from $0$ is always to $k$, $\mu_0(k)=2k$ is also the number of visits to $k$, starting from $k$, before seeing $0$.
Because this random walk agrees with the simple random walk on $\mathbb Z$ as long as we stay in the states $\{1,2,\dots\}$, $\mu_0(k)=2k$ is also the number of visits to $k$, starting from $k$, before seeing $0$ in the simple random walk.