The central limit theorem establishes that the average of $n$ i.i.d. random variables tends to a normal law, with parameters $\mu$ and $\sigma^2/n$.
The average is a non-biaised estimator of the mean of the distribution. If we turn to the case of the non-biased estimator of the distribution variance, $s^2$, the central limit theorem can also be used as $s^2$ is the difference of
-
the average of $n$ i.i.d. random variables ($\overline{x^2}$),
-
the square of a normally distributed variable (${\overline x}^2$).
Anyway the computation is uneasy as these two variables aren't independent.
As $s^2$ is unbiaised, its mean is $\sigma^2$. But what is the variance of $s^2$ ? And is its distribution normal in the limit ?
Best Answer
Let $s^2=\frac{n}{n-1}\left(\overline{x^2}-\left(\overline x\right)^2\right)=\frac{1}{n-1}\sum_{i=1}^n \left(x_i-\overline x\right)^2$. Define centered r.v.'s $y_i=x_i-\mathbb Ex_1$ and rewrite sample variance in terms of this r.v.'s: $$ s^2=\frac{1}{n-1}\sum_{i=1}^n \left(y_i-\overline y\right)^2 = \frac{n}{n-1}\left(\overline{y^2}-\left(\overline y\right)^2\right)=\overline{y^2}-\left(\overline y\right)^2+\frac{s^2}{n}. $$ Note that $\sigma^2=\text{Var}(x_1)=\mathbb E[y_1^2]$.
Find the limiting distribution of $\sqrt{n}\left(s^2-\sigma^2\right)$: $$\tag{1}\label{1} \sqrt{n}\left(s^2-\sigma^2\right) = \sqrt{n}\left(\overline{y^2}-\left(\overline y\right)^2+\frac{s^2}{n} -\sigma^2 \right)=\sqrt{n}\left(\overline{y^2}-\sigma^2 \right) -\sqrt{n}\left(\overline y\right)^2+\sqrt{n}\frac{s^2}{n} . $$ Next prove that $\sqrt{n}\left(\overline y\right)^2 \xrightarrow{p} 0$ and $\sqrt{n}\,\dfrac{s^2}{n}=\dfrac{s^2}{\sqrt{n}}\xrightarrow{p} 0$ as $n\to\infty$. Indeed, by Slutsky's theorem, $$\sqrt{n}\left(\overline y\right)^2 = \underbrace{\overline y}_{\begin{array}$\downarrow p \cr 0\end{array}} \cdot \underbrace{\sqrt{n}\left(\overline y\right)}_{\begin{array}$\downarrow d \cr N(0,1)\end{array}}\xrightarrow{d} 0\cdot N(0,1)=0$$ The convergence in distribution to zero implies the convergence in probability.
Next, $$ \dfrac{s^2}{\sqrt{n}} = s^2\cdot \frac{1}{\sqrt{n}}\xrightarrow{p} \sigma^2\cdot 0=0.$$ We obtain that the second and third terms in r.h.s. of (\ref{1}) tends to zero in probability. Consider the first term: $$ \sqrt{n}\left(\overline{y^2}-\sigma^2 \right) = \sqrt{n}\left(\overline{y^2}-\mathbb E\left[y_1^2\right] \right) \xrightarrow{d} N(0,\text{Var}(y_1^2))=N(0,\mathbb E\left[y_1^4\right]-\sigma^4). $$ By Slutsky's theorem, $$\tag{2}\label{2} \sqrt{n}(s^2-\sigma^2)\xrightarrow{d}N(0,\mathbb E\left[y_1^4\right]-\sigma^4)= N(0,{\mathbb E}\left[(x_1-\mathbb Ex_1)^4\right]-\sigma^4). $$ So, you can say that the limiting distribution of $s^2$ is normal with mean $\sigma^2$ and variance $$\dfrac{{\mathbb E}\left[(x_1-\mathbb Ex_1)^4\right]-\sigma^4}{n}.$$ But this words are extremely non-rigorous. The rigorous statement is (\ref{2}).