I am currently trying to come to grips with Markov Chains. I am confused by the nature of having a null recurrent state, which by definition is also recurrent. Say we have a Markov Chain starting at some $y\in E$ with a finite state space $E$. We know that $x$ is recurrent if and only if:
$P^{y}(\tau_{x} < \infty)=1$ where $\tau_{x}:=\inf\{n \in \mathbb N:X_{n}=x\}$ can be seen as the entry time of the Markov Chain into state $x$.
and thus $X_{n}=x$ for infinitely many $n \in \mathbb N$ and therefore the expected value of these visits to state $x$, i.e. $\mathbb E^{y}[\sum_{n \geq 0}1_{\{X_{n}=x\}}]=\infty$
Then, by definiton, this same recurrent state $x$ can be considered null recurrent if the expected mean recurrence time is infinite, i.e. $\mathbb E^{x}[\tau_{x}]=\infty$.
This seems extremely counterintuitive: null recurrence basically explains that the expected mean recurrence time to state $x$ of the Markov Chain is infinite. But since $x$ is recurrent, $X_{n}=x$ for infinitely many $n$ and $\tau_{x} < \infty$ almost surely.
In other words,
If a random variable $X$ is finite then how can its expectation $\mathbb E [X]$ be infinite?
Any explanations, and intuitions would be greatly appreciated.
Best Answer
As others have stated, there is no way for a recurrent state in a finite-state Markov chain to be null recurrent. It is possible in an infinite-state Markov chain; for example, in the simple $50$-$50$ random walk on the integers, all the states are null recurrent. (This was in saulspatz's original answer, but it was edited out.)
As to the general question of how a random variable can take on values that are finite, and yet have infinite expectation: That is indeed counter-intuitive, and yet there are undeniably examples of such random variables. One such is involved in the St Petersburg paradox. None of the potential rewards is infinite, but the expected value is.
Here's another example: Let $X$ be a discrete random variable on $\mathbb{N}$, such that
$$ P(X = k) = \frac{6}{\pi^2k^2} \qquad k \geq 1 $$
This is a probability distribution, by the Basel problem, but the expectation is
\begin{align} E(X) & = \sum_{k=1}^\infty kP(X = k) \\ & = \frac{6}{\pi^2} \sum_{k=1}^\infty \frac1k \\ & = \infty \end{align}
Strange but true!