Let
- $(E,\mathcal E,\nu)$ be a probability space
- $\kappa$ be a Markov kernel on $(E,\mathcal E)$ and $\operatorname P_x$ denote the unique probability measure on $\left(E^{\mathbb N},\mathcal E^{\otimes\mathbb N}\right)$ with $$\operatorname P_x\circ\:\pi_{\left\{1,\:\ldots\:,\:n\right\}}^{-1}=\nu\otimes\kappa^{\otimes(n-1)}\;\;\;\text{for all }n\in\mathbb N\tag1,$$ where $\pi_I$ is the projection of $E^{\mathbb N}$ onto $E^I$ for $I\subseteq\mathbb N$, for $x\in E$
- $(Z_n)_{n\in\mathbb N}$ be an ergodic time-homogeneous Markov chain on $(E,\mathcal E)$ with stationary distribution $\nu$
- $E_0\in\mathcal E$
- $\tau_0:=0$ and $$\tau_k:=\inf\left\{n>\tau_{k-1}:Z_n\in E_0\right\}\;\;\;\text{for }k\in\mathbb N$$
Assuming that $$\operatorname P_x\left[\inf\left\{n\in\mathbb N:\pi_{\{n\}}\in E_0\right\}<\infty\right]=1\;\;\;\text{for all }x\in E_0,\tag2$$ we know that $\left(Z_{\tau_k}\right)_{k\in\mathbb N}$ is again an ergodic time-homogeneous Markov chain with stationary distribution $$\nu_0:=\frac{\left.\nu\right|_{E_0}}{\nu(E_0)}.$$ Let $$N_n:=\sum_{i=1}^n1_{E_0}(Z_i)$$ denote the number of visits to $E_0$ up to time $n\in\mathbb N$.
Let $f\in\mathcal L^1(\nu)$. We know that $$\frac1n\sum_{i=1}^nf(Z_i)\xrightarrow{n\to\infty}\int f\:{\rm d}\nu\tag3$$ and $$\frac1n\sum_{i=1}^nf\left(Z_{\tau_i}\right)\xrightarrow{n\to\infty}\int f\:{\rm d}\nu_0\tag4.$$ To what do $$\frac1n\sum_{i=1}^{N_n}f\left(Z_{\tau_i}\right)\tag5$$ and $$\frac1{N_n}\sum_{i=1}^{N_n}f\left(Z_{\tau_i}\right)\tag6$$ converge?
Best Answer
Let $g(x) = f(x) \chi_{E_0}(x)$, then $$ \frac1n\sum_{i=1}^{N_n} f(Z_{\tau_i}) = \frac1n \sum_{i=1}^n g(Z_i) $$ since the number of nonzero terms on the RHS is $N_n$ and whenever $x\in E_0$, $f(x)=g(x)$. Thus $$ \frac1n \sum_{i=1}^{N_n} f(Z_{\tau_i}) = \frac1n \sum_{i=1}^n g(Z_i) \to \int g d\nu = \int_{E_0} fd\nu = \nu(E_0) \int f d\nu_0 \quad \mbox{a.s.} $$
Note that $(\frac1{N_n} \sum_{i=1}^{N_n} f(Z_{\tau_i}))_n$ is a subsequence of ($ \frac1n\sum_{i=1}^n f(Z_{\tau_i}))_n$, since $N_n<N_{n+1}$ a.s. Hence $$ \frac{1}{N_n}\sum_{i=1}^{N_n} f(Z_{\tau_i}) \to \lim_n \frac1n \sum_{i=1}^n f(Z_{\tau_i}) = \int f d\nu_0 \quad \mbox{a.s.} $$
Update on your comment: $$\frac{N_n}{n} = \frac1{n}\sum_{i=1}^n \mathbf 1_{E_0} (Z_i) \to \int \mathbf 1_{E_0} d\nu = \nu(E_0) $$ hence $$\frac{1}{N_n}\sum_{i=1}^n g(Z_i) = \frac{n}{N_n}\frac1n \sum_{i=1}^n g(Z_i) \to \frac1{\nu(E_0)} \int gd\nu $$ hence $$ \frac{\sum_{i=1}^{N_n}f(Z_{\tau_i})}{\sum_{i=1}^n g(Z_i)} \to \frac{\int fd\nu_0}{\frac{1}{\nu(E_0)}\int gd\nu} = \frac{\int f d\nu}{\int g d\nu} = \frac{\int fd\nu}{\int_{E_0} f d\nu} $$