[Math] Central limit theorem confusion

Question

If a bunch of random variables $X_i$ are independently and identically distributed with an exponential distribution, their sum apparently follows a Gamma distribution.

But doesn't the central limit theorem imply that (for $X_i$ of any distribution with mean zero and variance $\sigma^2$), the sum $\sum_{i=1}^n X_i$ will become approximately normally distributed $~N(0,n\sigma^2)$ for large enough $n$ ?

Obviously I am missing something basic, but what's going on? How can the sum of i.i.d. exponential random variables have a Gamma distribution, but also be converging to normality?

Answer 1

There are several confusions here (I was also very confused when I started learning about that topic :-).)

• Exponential random variables have a non zero mean (and are positive). The quantity you should be looking at, which asymptotically converges in distribution to a normal variable is $$\sqrt{n} \left( \frac{\sum_{i = 1}^n X_i}{n} - \mu \right)$$ The $\sqrt{n}$ was essential here, otherwise the distribution of the average will converge to a point mass at $\mu$. That quantity will converge to $N(0,\sigma^2)$. Both $\mu$ and $\sigma$ will be determined by the parameter of the exponential distribution.
• The central limit theorem is asymptotic. The quantity $\sqrt{n} \left( \frac{\sum_{i = 1}^n X_i}{n} - \mu \right)$ will have a distribution. Let's call it $F_n$. (it is essential to remember that it depends on $n$). $F_n$ in general is not a normal distribution $N(0,\sigma^2)$. The central limit theorem tells us that that distribution gets in a certain sense closer and closer to $N(0,\sigma^2)$ as $n \to \infty$.