Let $X_0, X_1, X_2, \ldots$ be a sequence of i.i.d. real-valued random variables on some probability space $(\Omega, \mathcal{F}, \mathbb{P})$ with continuous CDF $F(x)$ and define a sequence of empirical CDFs $F_n(x) = \frac{1}{n} \sum_{i = 1}^n \mathbf{1}_{\{X_i \leq x\}}(x)$ ($\mathbf{1}_A$ is an indicator function). It is well known that
$$\sup_{x \in \mathbb{R}} \left|F_n(x) – F(x)\right| \to 0 \text{ a.s.}$$
(This is the fundamental theorem of mathematical statistics, or FTMS, supposedly.) Are we then safe to say that
$$F_n(X_0) \to U \text{ in law}$$
where $U$ follows a standard uniform distribution? Or could we make the even stronger claim that we can redefine the random variables into a probability space such that the convergence happens almost surely?
It seems like this would be the case. FTMS suggests we can write $F_n(X_0) = F(X_0) + o(1)$ (holding almost surely) and the distribution of $F(X_0)$ is a standard uniform distribution. While $F_n$ is affected by the asymptotics, $X_0$ is not. The probability $X_0$ takes a value that is not covered by the convergence is zero thanks to FTMS being a uniform convergence result. Is it that easy? Am I overthinking this?
Best Answer
$\newcommand\Om\Omega\newcommand\om\omega\newcommand\R{\mathbb R}$This is indeed straightforward. Spelling out $$\sup_{x\in\R}|F_n(x)-F(x)|\to0 \text{ a.s.}, $$ we see that for some subset $N$ of $\Om$ of outer probability $0$ and all $\om\in\Om_0:=\Om\setminus N$ we have $$\sup_{x\in\R}\Big|\frac1n\sum_{j=1}^n 1(X_j(\om)\le x)-F(x)\Big|\to0$$ and hence $$\frac1n\sum_{j=1}^n 1(X_j(\om)\le X_0(\om))\to F(X_0(\om)).$$
So, indeed we have $$F_n(X_0)\to F(X_0)$$ a.s. and hence in law. Also, if $F$ is continuous, then the random variable $F(X_0)$ has the standard uniform distribution.