Almost sure uniform convergence of the empirical distribution

probabilityprobability distributionsstatisticsuniform distributionuniform-convergence

Let $U_1, U_2, \dots \overset{\text{iid}}{\sim} U[0,1]$, i.e. a sequence of independent and identically distributed uniform random variables on the set $[0,1]$.

By the Strong Law of Large Numbers, we know that for any $t \in [0,1]$
$$
\left|\frac{1}{n} \sum_{j=1}^n \mathbf{1}\{U_j \leq t\} – t\right| \to 0 \quad \text{almost surely},
$$

since $\mathbb{E}(\mathbf{1}\{U_j \leq t\}) = \mathbb{P}(U_j \leq t) = t$. Thus, we have pointwise almost sure convergence.

How can we then show that uniform almost sure convergence also holds? That is
$$
\sup_{0 \leq t \leq 1}\left|\frac{1}{n} \sum_{j=1}^n \mathbf{1}\{U_j \leq t\} – t\right| \to 0 \quad \text{almost surely}.
$$

Best Answer

The following result is found in many books, in particular K L Chung's book:

If $F_n,F$ are distribution functions such that $F_n(t) \to F(t)$ on dense set of points $t$ and if $F$ is continuous then $F_n \to F$ uniformly.

In our case there is a null set outside which $F_n(t) \to F(t)$ for all $t$ rational. Hence the result follows.