In order for the CLT to hold we need the distribution we wish to approximate to have mean $\mu$ and finite variance $\sigma^2$. Would it be true to say that for the case of the Cauchy distribution, the mean and the variance of which, are undefined, the Central Limit Theorem fails to provide a good approximation even asymptotically?
Probability – Exploring Cauchy Distribution and the Central Limit Theorem
asymptoticscauchy distributioncentral limit theoremprobability
Related Solutions
The OP says
The central limit theorem states that the mean of i.i.d. variables, as N goes to infinity, becomes normally distributed.
I will take this to mean that it is the OP's belief that for i.i.d. random variables $X_i$ with mean $\mu$ and standard deviation $\sigma$, the cumulative distribution function $F_{Z_n}(a)$ of $$Z_n = \frac{1}{n} \sum_{i=1}^n X_i$$ converges to the cumulative distribution function of $\mathcal N(\mu,\sigma)$, a normal random variable with mean $\mu$ and standard deviation $\sigma$. Or, the OP believes minor re-arrangements of this formula, e.g. the distribution of $Z_n - \mu$ converges to the distribution of $\mathcal N(0,\sigma)$, or the distribution of $(Z_n - \mu)/\sigma$ converges to the distribution of $\mathcal N(0,1)$, the standard normal random variable. Note as an example that these statements imply that $$P\{|Z_n - \mu| > \sigma\} = 1 - F_{Z_n}(\mu + \sigma) + F_{Z_n}((\mu + \sigma)^-) \to 1-\Phi(1)+\Phi(-1) \approx 0.32$$ as $n \to \infty$.
The OP goes on to say
This raises two questions:
- Can we deduce from this the law of large numbers? If the law of large numbers says that the mean of a sample of a random variable's values equals the true mean μ as N goes to infinity, then it seems even stronger to say that (as the central limit says) that the value becomes N(μ,σ) where σ is the standard deviation.
The weak law of large numbers says that for i.i.d. random variables $X_i$ with finite mean $\mu$, given any $\epsilon > 0$, $$P\{|Z_n - \mu| > \epsilon\} \to 0 ~~ \text{as}~ n \to \infty.$$ Note that it is not necessary to assume that the standard deviation is finite.
So, to answer the OP's question,
The central limit theorem as stated by the OP does not imply the weak law of large numbers. As $n \to \infty$, the OP's version of the central limit theorem says that $P\{|Z_n-\mu| > \sigma\} \to 0.317\cdots$ while the weak law says that $P\{|Z_n-\mu| > \sigma\} \to 0$
From a correct statement of the central limit theorem, one can at best deduce only a restricted form of the weak law of large numbers applying to random variables with finite mean and standard deviation. But the weak law of large numbers also holds for random variables such as Pareto random variables with finite means but infinite standard deviation.
I do not understand why saying that the sample mean converges to a normal random variable with nonzero standard deviation is a stronger statement than saying that the sample mean converges to the population mean, which is a constant (or a random variable with zero standard deviation if you like).
Yes, the sample standard deviation is asymptotically normal. Let the sample standard deviation be $\hat{\sigma} = \sqrt{\frac{1}{n} \sum_{i=1}^n (x_i - \bar{x})^2}$, and let $\sigma$ be the population standard deviation. Let's use the central limit theorem to show that $$ \sqrt{n}(\hat{\sigma} - \sigma) \xrightarrow{d} N(0, V). $$ First write things as $$ \sqrt{n}(\hat{\sigma} - \sigma) = \sqrt{n}\left(\sqrt{ \frac{1}{n} \sum_{i=1}^n (x_i - \bar{x})^2} - \sqrt{\sigma^2} \right)$$ The central limit theorem tells is about how sample moments minus population moments behave. If we didn't have square roots above, we'd just have something like sample moments minus population ones, and we could use the central limit theorem. To get rid of the square roots, let's take a Taylor expansion of the first square root around $\sigma^2$. $$ \sqrt{ \frac{1}{n} \sum_{i=1}^n (x_i - \bar{x})^2} = \sqrt{\sigma^2} + \frac{1}{2\sigma} \left( \frac{1}{n} \sum_{i=1}^n (x_i - \bar{x})^2 - \sigma^2\right) + O\left(\left( \frac{1}{n} \sum_{i=1}^n (x_i - \bar{x})^2 - \sigma^2\right)^2\right) $$ Plugging this into the above, we have $$\sqrt{n}(\hat{\sigma} - \sigma) = \frac{\sqrt{n}}{2 \sigma} \left( \frac{1}{n} \sum_{i=1}^n (x_i - \bar{x})^2 - \sigma^2\right) + O\left(\frac{1}{n^{3/2}} \left(\sum_{i=1}^n (x_i - \bar{x})^2 - \sigma^2\right)^2\right)$$ Rearranging the first term on the right gives $$\frac{\sqrt{n}}{2 \sigma} \left( \frac{1}{n} \sum_{i=1}^n (x_i - \bar{x})^2 - \sigma^2\right) = \frac{1}{2\sigma \sqrt{n}} \sum_{i=1}^n ((x_i - E[X])^2 - \sigma^2) - \frac{\sqrt{n}}{2\sigma}(\bar{x} - E[x])^2 $$ The central limit theorem tells us that under some conditions $\frac{1}{\sqrt{n}} \sum_{i=1}^n (y_i - E[y]) \xrightarrow{d} N(0, \sigma_y^2)$. With $y_i = (x_i - E[x])^2$ and $E[y] = \sigma^2$, then the first term on the right converges in distribution to a normal.
The second term we can write as $\sqrt{n}(\bar{x} - E[x]) (\bar{x} - E[x])$, and since $\sqrt{n}(\bar{x} - E[x]) \xrightarrow{d} N$ and $(\bar{x} - E[x]) \xrightarrow{p} 0$, by Slutsky's lemma, the product converges in probability to 0.
Similarly, we could show that, $\frac{1}{n^{3/2}} \left(\sum_{i=1}^n (x_i - \bar{x})^2 - \sigma^2\right)^2 \xrightarrow{p} 0$, so the remainder from the Taylor expansion vanishes.
This Taylor expansion trick comes up often, so it has a name. It's called the delta method.
Best Answer
The distribution of the mean of $n$ i.i.d. samples from a Cauchy distribution has the same distribution (including the same median and inter-quartile range) as the original Cauchy distribution, no matter what the value of $n$ is.
So you do not get either the Gaussian limit or the reduction in dispersion associated with the Central Limit Theorem.