For distribution function on $R$, $F(x)=P(X\le x)$, a natural estimator is empirical CDF $F_n(x)=\frac{1}{n}\sum I[X_i\le x]$. From the Strong law of large number,
$$
F_n\to F
$$
almost surely. Then $F_n\to F$ in probability because convergence almost surely imply convergence in probability. That means $F_n$ is a weak consistent estimator of $F$, right?
Am I right for the above result?
But for general empirical measure, do we have similar result?
For a simple case, let random variables $X_1,\dots, X_n$ sample from Borel probability measure $\mu((-\infty,x])=F(x)$ on $R$. Let $\mu_n=\frac{1}{n}\sum \delta_{X_i}$ be the empirical measure of $\mu$. Is there a result about
$$
\mu_n\to \mu
$$
convergence weakly?
Best Answer
Let $(X_n)_{n \in \mathbb{N}}$ be real-valued IID samples with law $\mu:\mathscr{B}(\mathbb{R})\to [0,1]$. We have $$\begin{aligned}\int_\mathbb{R}e^{i\xi x}\mu_n(dx,\omega)&=\frac{1}{n}\sum_{k\leq n}\int_\mathbb{R}e^{i\xi x}\delta_{X_k(\omega)}(dx)=\\ &=\frac{1}{n}\sum_{k\leq n}e^{i\xi X_k(\omega)}\stackrel{n \to \infty,\textrm{ a.s.}}{\to}E[e^{i\xi X_1}]=\int_{\mathbb{R}}e^{i\xi x}\mu(dx),\,\quad \forall \xi \in \mathbb{R}\end{aligned}$$ by LLN. This establishes $\mu_n(\omega)\to\mu$ weakly, $P$-a.s.