Solved – Are there any examples of where the central limit theorem does not hold

central limit theoremmathematical-statisticsnormal distributionprobability

Wikipedia says –

In probability theory, the central limit theorem (CLT) establishes that, in most situations, when independent random variables are added, their properly normalized sum tends toward a normal distribution (informally a "bell curve") even if the original variables themselves are not normally distributed…

When it says "in most situations", in which situations does the central limit theorem not work?

Best Answer

To understand this, you need to first state a version of the Central Limit Theorem. Here's the "typical" statement of the central limit theorem:

Lindeberg–Lévy CLT. Suppose ${X_1, X_2, \dots}$ is a sequence of i.i.d. random variables with $E[X_i] = \mu$ and $Var[X_i] = \sigma^2 < \infty$. Let $S_{n}:={\frac {X_{1}+\cdots +X_{n}}{n}}$. Then as $n$ approaches infinity, the random variables $\sqrt{n}(S_n − \mu)$ converge in distribution to a normal $N(0,\sigma^2)$ i.e.

$${\displaystyle {\sqrt {n}}\left(\left({\frac {1}{n}}\sum _{i=1}^{n}X_{i}\right)-\mu \right)\ {\xrightarrow {d}}\ N\left(0,\sigma ^{2}\right).}$$

So, how does this differ from the informal description, and what are the gaps? There are several differences between your informal description and this description, some of which have been discussed in other answers, but not completely. So, we can turn this into three specific questions:

  • What happens if the variables are not identically distributed?
  • What if the variables have infinite variance, or infinite mean?
  • How important is independence?

Taking these one at a time,

Not identically distributed, The best general results are the Lindeberg and Lyaponov versions of the central limit theorem. Basically, as long as the standard deviations don't grow too wildly, you can get a decent central limit theorem out of it.

Lyapunov CLT.[5] Suppose ${X_1, X_2, \dots}$ is a sequence of independent random variables, each with finite expected value $\mu_i$ and variance $\sigma^2$ Define: $s_{n}^{2}=\sum _{i=1}^{n}\sigma _{i}^{2}$

If for some $\delta > 0$, Lyapunov’s condition ${\displaystyle \lim _{n\to \infty }{\frac {1}{s_{n}^{2+\delta }}}\sum_{i=1}^{n}\operatorname {E} \left[|X_{i}-\mu _{i}|^{2+\delta }\right]=0}$ is satisfied, then a sum of $X_i − \mu_i / s_n$ converges in distribution to a standard normal random variable, as n goes to infinity:

${{\frac {1}{s_{n}}}\sum _{i=1}^{n}\left(X_{i}-\mu_{i}\right)\ {\xrightarrow {d}}\ N(0,1).}$

Infinite Variance Theorems similar to the central limit theorem exist for variables with infinite variance, but the conditions are significantly more narrow than for the usual central limit theorem. Essentially the tail of the probability distribution must be asymptotic to $|x|^{-\alpha-1}$ for $0 < \alpha < 2$. In this case, appropriate scaled summands converge to a Levy-Alpha stable distribution.

Importance of Independence There are many different central limit theorems for non-independent sequences of $X_i$. They are all highly contextual. As Batman points out, there's one for Martingales. This question is an ongoing area of research, with many, many different variations depending upon the specific context of interest. This Question on Math Exchange is another post related to this question.

Related Question