So I know nothing about large deviations theory, and I'm reading some notes. They claim that:
The CLT does not hold far away from the peak
I am not sure how to parse this statement. There are many statements of the CLT but here is the one I know:
Let $X_n$ be a sequence of i.i.d. random variables with mean $0$ and variance $\sigma^2<\infty$. Then the following sum:
$$\frac{1}{\sqrt{n}}\sum_{k=1}^n X_n$$
converges in distribution as $n\to\infty$ to $N(0,\sigma^2)$.
Why do the notes say that central limit theorem doesn't hold away from $0$? There's nothing in the central limit theorem that says "only for some interval around $0$". Does it just mean that the convergence rate is very slow and impractical?
Best Answer
Well, essentially it means that if you are far away from the mean (which in this case is 0), the 'approxmation' that the sum random variable (call it r.v. $Y$) you have is like the normal distribution becomes really bad.
If you want to understand this, just take some simple example. Let $n=10$ and say you have uniform random variables in range $[0,1]$. If you calculate something like $P(10000<Y<1000000)$ twice, once with the real distribution of $Y$ and another time with the normal 'approximation', you shall see that the difference is huge! Does that make sense?