[Math] Markov chain: closed, finite classes are recurrent

markov chains

In Norris: Markov Chains the closed class C is defined as one for which $i\in C$ and $P_i(X_n=j \text{ for some }n\ge0)>0$ implies that $j\in C$.

Here's theorem 1.5.6 from the book with proof

Every finite closed class is recurrent.

Proof: Suppose C is closed and finite and that the chain $X_n$ starts in $C$. Then for some $i\in C$ we have

$$0<P(X_n =i\text{ for infinitely many }n)=P(X_n =i \text{ for some } n)P_i(X_n =i\text{ for infinitely many }n)$$

by the strong Markov property. So $i$ is not transient, hence $C$ is recurrent.

I'm confused because take for example a state the chain never reaches, say $y$. Then the union of an irreducible set (where all pairs of states communicate) and $y$ is not irreducible anymore but it is closed. So $y$ cannot be recurrent since it is never reached. Hence the irreducibility condition might be needed in addition closed + finiteness to imply recurrence?

Best Answer

The word "class" as defined in the book means "comunicating class" (i.e. class = irreducible set). Since you have added the state $y$ it is not anymore comunnicating since it is unreachable and thus the property does not apply.

Related Question