Why does the time average of a positive recurrent Markov chain converge to the stationary space average

markov chainsprobability theoryreference-requeststochastic-processes

Result

Let $X$ be a positive recurrent irreducible continuous time Markov chain taking values in $\mathbb Z$ with stationary distribution $\pi$. Let $\mathbb E_\mu$ denote expectation when the process is started with initial distribution $\mu$. Then

$$\lim_{t\to\infty}\frac1t\int_0^t X(s)\,\mathrm ds = \mathbb E_\pi[X(0)]$$

almost surely regardless of the initial distribution of the chain, provided $\mathbb E_\pi[|X(0)|]<\infty$.

Attempts

I searched this site using Approach Zero. The closest I found was a comment on this answer, but it doesn't provide a proof.

I checked Rick Durrett's Probability: Theory and Examples, but he doesn't cover continuous time Markov chains.

I checked Reversibility in Stochastic Networks by Frank Kelly, but the closest he gets appears to be stating "the proportion of time the process spends in state $k$ during the period $[0,t]$ converges to $\pi(k)$ as $t\to\infty$" without proof.

I checked Markov Chains by J. R. Norris. His Ergodic Theorem (3.8.1) is that for any bounded function $f$ we have
$$\lim_{t\to\infty}\frac1t\int_0^t f(X(s))\,\mathrm ds = \mathbb E_\pi[f(X(0))]$$
almost surely. But the identity function isn't bounded.

Comments

I'm convinced this result is (a) true and (b) part of the standard theory of such chains. But I can't find a reference. Either a reference that proves this result exactly, or a reference to some more general (ergodic?) theorem plus a careful derivation of this result from that one would be an acceptable answer.

Best Answer

Theorem 45 in chapter 4 of Richard Serfozo's Basics of Applied Stochastic Processes says that any ergodic CTMC $X(t)$ with stationary distribution $p$, and any function $f:S\to\mathbb R$,

$$\lim_{t\to\infty}\frac1t\int_0^t f(X(s))\,\mathrm ds = \sum_j f(j)p_j$$

a.s. provided the sum is absolutely convergent.

Since the text defines an ergodic CTMC as one which is irreducible and positive recurrent, in the language of the question this says that as long as the chain is irreducible and $\mathbb E_\pi[|X(0)|]<\infty$ then

$$\lim_{t\to\infty}\frac1t\int_0^t X(s)\,\mathrm ds = \mathbb E_\pi[X(0)],$$

which is what I wanted.

(I think I had the assumptions of irreducibility and finite mean in mind when I wrote the question, but I clearly didn't include them. I'll edit the question.)

Related Question