I know the statement in the title is only true for finite-state Marove chains: if the chain has infinitely many states, all states can be null recurrent, e.g., one-dimensional symmetric random walks. However, I don't see where the proof goes wrong with countably many states. Can you help me point it out?
Let us suppose the Markov chain is irreducible and recurrent with state space $S$, which can be either finite or countably infinite. According to (6.6.1) in Durrett's textbook, for any $x,y \in S$, we have
$$
\frac{1}{n}\sum_{m=1}^{n}p^{m}(x,y) \overset{n\to \infty}{\longrightarrow} \frac{1}{\mathbb{E}_{y}[T_y]}, \tag{1}\label{1}
$$
where $p^m(x,y)$ is the $m$-step transition probability and $T_y$ is the time of the first return to state $y$.
Suppose all states are null recurrent, i.e., $\mathbb{E}_{y}[T_y] = \infty$ for all $y \in S$. Sum over $y \in S$ on both sides of (\ref{1}).
On the left-hand side, we have for any $n$,
$$
\sum_{y\in S}\frac{1}{n}\sum_{m=1}^{n}p^{m}(x,y) = \frac{1}{n}\sum_{m=1}^{n}\sum_{y\in S}p^{m}(x,y) = 1.
$$
On the right-hand side, we have
$$\sum_{y\in S} \frac{1}{\mathbb{E}_{y}[T_y]} = \sum_{y\in S} 0 = 0.$$
Thus above implies that $1 \to 0$, which is impossible. Thus we must have a positive recurrent state.
I was wondering what is wrong with the above statement?
Best Answer
First of all, 6.6.1 in Durrett says $$\frac{1}{n}\sum_{m=1}^n p^m(x,y) \to \frac{1}{E_yT_y} 1_{\{T_y<\infty\}}.$$ Second of all, the sum over $y$ does not necessarily commute with the limit if there are infinitely many $y$'s. The essence of this statement is that for each $y$ and each $n$, we have some value $a_n(y)$, and $a_n(y) $ converges to some $a(y)$ as $n \to \infty$. You want to conclude that $$\sum_{ y} a_n(y) \to \sum_{y} a(y)$$ but this is not necessarily the case.
For instance, suppose that the $y$'s take on values in $\mathbb{N}$ and $a_n(y) = \frac{1}{n+y}$. Then $a_n(y) \to 0$ for each $y$, but for any fixed value of $n$ we have $$\sum_y a_n(y) = \frac{1}{n+1}+\frac{1}{n+2}+\frac{1}{n+3}+\cdots = \infty.$$