I am working on a problem. Show that for each probability measure $\mu$, there exists probability measure $\mu_n$ with finite support such that $\mu_n$ converges weakly to $\mu$. I am thinking about the empirical measure, which has the distribution function $F_n(t)=1/n\sum 1(X_i\le t)$. So from LLN, we have $F_n(t)\rightarrow F(t)$ for each fixed $t$ a.s. But the convergence here is almost surely. So does this still means $F_n$ converges to $F$ weakly?
[Math] Weak convergence of probability measure
measure-theoryprobabilityprobability theory
Best Answer
Sorry if this is not exactly an answer to your original question. But I get a little bit itchy when I see the way some probabilists treat weak convergence of measures...
The Riesz Representation Theorem states that when $X$ is locally compact Hausdorff, we have an isometry between the dual of $C_0(X)$ and the normed vector space of signed finite regular measures on the Borel sets of $X$. The norm of a measure is its total variation. In this context, weak convergence of measures is simply the weak-* convergence in $C_0(X)$.
The Banach-Alaoglu Theorem states that the closed unit ball is compact in the weak-* topology. It is very easy to show that the set of (positive) measures such that $0 \leq \mu(X) \leq 1$ is a closed (in the weak-* topology) subset of the unit ball, and therefore is also compact. Let's represent the set of those measures by $\mathcal{M}$. Notice that if $X$ is not compact, then the set of probability measures (on the Borel sets of $X$) is not compact in the weak-* topology. In fact, if we take a sequence $x_n \in X$ "convergin to $\infty$", then $$ \frac{1}{n} \sum_{j=1}^n \delta_{x_n} \rightarrow 0. $$
Since $\mathcal{M}$ is compact and convex, the Krein-Milman Theorem says that $\mathcal{M}$ is the weak-* closure of the convex combination of extremal points of $\mathcal{M}$. The extremal points are points that are not non-trivial convex combination of other points of $\mathcal{M}$. Notice that these are the measure $0$, and the Dirac deltas $\delta_x$. Therefore, the convex combinations of those are the measures $\mu_n$ with finite support. Now, it only remains to show that we can use $\frac{1}{\mu_n(X)} \mu_n$ instead of $\mu_n$, to conclude that any probability measure is the weak-* limit of probability measures with finite support.
I realize that this is quite complicated. But IMHO, concepts borrowed from functional analysis should be regarded as such. Of course, this is a matter of taste, but the definition using $F_n(t)$ is too artificial EVEN when compared to the definition using the weak-* topology on $C_0(X)$.