Let's say that we have a complete and separable metric space in other words a Polish space $(S, d)$.
If we say that we have $n$ I.I.D random elements $(Y_0, Y_1 …)$ in our space $S$ with a common distribution that we'll denote $P$. If we define an empirical measure $P_{n,w}$ using observations $(Y_1(w), …, Y_n(w))$.
$$P_{n,w} = \frac{1}{n}\sum_{i=1}^{n}\delta_{X_i}(w)$$
Where the $\delta_x$ is a measure that places a unit mass on $x$. How do we show that given that S is seperable and complete, then $P_{n,w} \rightarrow P$ as $n$ goes to $\infty$ The convergence is weak.
Could we use the equivalences from Portmanteau theorem's to prove this?
Best Answer
For each continuous bounded function $g: S\to\mathbb{R}$, we have $$\lim_n\int g~\mathrm dP_{n,w}=\int g~\mathrm dP$$ for $P$-almost all $w$ by the strong law of large numbers. This is not enough because there is a continuum of such functions and the union of all the relevant null sets may have positive measure. But there exist countably many bounded continuous functions such that convergence of their integrals implies already convergence for all bounded continuous functions. This is basically equivalent to the separability of the topology of weak convergence of probability measures, which you can find in many places. If you want the original source, it is:
To make the connection with separability more explicit: The weak topology is generated by functions of the form $\mu\mapsto\int g~\mathrm d\mu.$ Therefore, every open set in the weak topology is the union of finite intersections of preimages of open sets of reals under such functions. Now in a metrizable space, being separable is equivalent to having a countable basis. So let $D$ be a countable dense set of probability measures and $\mathcal{B}$ a countable basis of the weak topology. There are countably many pairs $(\mu,O)$ with $\mu\in D$, $O\in\mathcal{B}$, and $\mu\in O$. For each such pair, there is a finite set of $G_{\mu,O}$ of bounded continuous functions of $S$ such that for some family $\{V_g\mid g\in G_{\mu,O}\}$ of open sets of reals, we have $$\mu\in\bigcap_{g\in G_{\mu,O}}g^{-1}(V_g)\subseteq O.$$ It follows that sets of the form $$\bigcap_{g\in G_{\mu,O}}g^{-1}(V_g)$$ form themselves a countable basis of the weak topology. Now each such set requires only finitely many bounded continuous functions of $S$ and the countable union of finite sets is countable again, so we can do with countably many such functions. Specifically, the sequence of probability measures $\langle \mu_n\rangle$ converges to $\mu$ if and only if $$\lim_n\int g~\mathrm d\mu_n=\int g~\mathrm d\mu$$ for each $g\in\bigcup_{(\mu,O)} G_{\mu,O}$.
Conversely, if we have such a countable family of functions, there is a countable basis consisting of finite intersections of preimages of intervals with rational endpoints under the induced functions of probability measures.