Let $(X,E,P)$ denote a Markov chain, where $X=(X_n)_{n\in\mathbb{N}_0}$, $E$ is finite state space and $P$ is the transition matrix. Claim: Every finite closed class is recurrent.
Here is how we proved that, where
$$
H(i):=\inf\left\{n\geq 0: X_n=i\right\}.
$$
Let $C$ be a finite closed class. Suppose that $X$ starts in $C$. Then there exists a state $i\in C$ which is visited infinitely often.
Then
\begin{align}
0&<\mathbb{P}(X_n=i\text{ for infinitely many n})\\&=\mathbb{P}(\limsup_{n\to\infty}\left\{X_n=i\right\})\\
&=\mathbb{P}(\limsup_{n\to\infty}\left\{X_n=i\right\},H(i)<\infty)\\
&=\mathbb{P}(\limsup_{n\to\infty}\left\{X_n=i\right\}|H(i)<\infty)\cdot\mathbb{P}(H(i)<\infty)
\end{align}
Because $H$ is a stopping time, one can apply the strong Markov property, that is
$$
\mathbb{P}(\limsup_{n\to\infty}\left\{X_n=i\right\}|H(i)<\infty)\cdot\mathbb{P}(H(i)<\infty)=\mathbb{P}_i(\limsup_{n\to\infty}\left\{X_n=i\right\})\cdot\mathbb{P}(H(i)<\infty).~~(*)
$$
Because $\mathbb{P}(H(i)<\infty)=1$, we have
$$
0<\mathbb{P}_i(\limsup_{n\to\infty}\left\{X_n=i\right\}),
$$
i.e. $i$ is recurrent. Because recurrence is a class property, $C$ is recurrent.
I have three questions to this proof:
1.) Why can we suppose that $X$ starts in $C$?
2.) If the chain starts in $C$ and $C$ is finite and closed: Why is there a state $i\in C$ that is visited infinitely often?
3.) Where is the finiteness needed?
4.) I do not understand how by the Strong Markov Property the identity $(*)$ does follow.
Best Answer
1) We can assume without loss of generality that $X$ starts in $C$, as otherwise we could consider the Markov chain $\{X_\tau, X_{\tau+1},\ldots\}$ where $\tau = \inf\{n\geqslant 0 : X_n\in C\}$.
2) If $X_0\in C$, then $X_n\in C$ for all $n$. Since $C$ is finite, if each state were visited only finitely many times, this would contradict the fact that $X_0, X_1, \ldots$ is an infinite sequence.
3) For a counterexample, a biased random walk on the integers is transient.
4) I am not sure exactly how to formalize this, but intuitively, given that $H(i)<\infty$, there exists an $n$ such that $X_n=i$. So the event $\limsup_{n\to\infty}\{X_n=i\}$ is independent of $X_1,\ldots, X_n$, and we can assume without loss of generality that $X_0=i$ in computing the probability.