My simulation of the Central Limit Theorem does not converge to correct value

central limit theoremprobability

The Lindeberg–Lévy CLT states:

Assume $\\{ X_1, X_2, \dots \\}$ is a sequence of i.i.d. random variables with $\mathbb{E}[X_i] = \mu$ and $\text{Var}[X_i] = \sigma^2 < \infty$. And let $S_n = \frac{X_1 + X_2 + \dots + X_n}{n}$. Then as $n$ approaches infinity, the random variables $\sqrt{n}(S_n − \mu)$ converge in distribution to a normal $\mathcal{N}(0, \sigma^2)$.

I have written a small numerical simulation to check my understanding, and it does not do what I hypothesized. Each red dot is the estimated $\sigma^2$ for 2000 samples of $\sqrt{n}(S_n – \mu)$ from the uniform distribution. The blue line is the true $\sigma^2$ for the uniform distribution. I would expect that as $n$ increases, the red dots converge to towards the blue line.

enter image description here

But that is not what happens. What's wrong with my understanding of the CLT? Here is the code that generated the figure.

import numpy as np
import matplotlib.pyplot as plt
from   scipy.stats import norm

a = 0
b = 1
mu_lim  = 1/2.  * (a + b)
var_lim = 1/12. * (b - a)**2
reps = 2000
fig, axes  = plt.subplots(1)
variances = []
x = range(1, 1000, 10)

for n in x:
    rvs = []
    for _ in range(reps):
        Sn = np.random.uniform(a, b, n).mean()
        rvs.append(np.sqrt(n) * (Sn - mu_lim))
    _, std = norm.fit(rvs)
    variances.append(std**2)

plt.plot(x, [var_lim for _ in x])
plt.scatter(x, variances)
plt.show()

Best Answer

It is not surprising that you don't see a convergence in your diagram because you always compute the variance on 2000 samples. When $n$ increase, you only approximate $\mathcal{N}(0, \sigma^2)$ better. Once $n$ is sufficiently large your experience basically becomes: Get 2000 sample with distribution $\mathcal{N}(0,\sigma^2)$ and print the experimental variance of the sample. But this experience does not depends on $n$ and hence the distribution of the experimental variance is always the same since you always make 2000 simulations.

Related Question