Given $X_1,…,X_n$ iid to a certain distribution (not necessarily normal), with $\mathbb{E}(X_i)=\mu$ and $\mathbb{V}(X_i)=\sigma^2$, I'm trying to deduce the standard and mean squared error of the estimator $\widehat{\sigma}^2=S_n^2$, where $S_n^2$ is the sample variance, given by$$
S_n^2=\frac{1}{n-1}\sum_{i=1}^n(X_i-\overline{X}_n)^2.
$$
To do so I need its variance, $\mathbb{V}(S_n^2)$. Since I know the expectation $\mathbb{E}(S_n^2)=\sigma^2$, I started by expanding$$
\mathbb{V}(S_n^2)=\mathbb{E}(S_n^4)-(\mathbb{E}(S_n^2))^2=\mathbb{E}(S_n^4)-\sigma^4
$$
but I'm stuck with the expansion of the term $\mathbb{E}(S_n^4)$. Any ideas?
P.S. – I saw user940's answer on this question, but I was looking for a different approach, also not assuming normal distributed random variables.
Best Answer
In the meantime I think I solved it. Here's my solution:
Let $\mu_k$ denote the $k$th central momentum of $X_i$, i.e, $\mu_k=\mathbb{E}((X_i-\mu)^k)$, and $Z_i\equiv X_i-\mu$ for all $i$. Thus $\mathbb{E}(Z_i)=0$. Since$$ \mathbb{V}(S_n^2)=\mathbb{E}(S_n^4)-(\mathbb{E}(S_n^2))^2=\mathbb{E}(S_n^4)-\sigma^4, $$we derive an expression of $\mathbb{E}(S_n^4)$ in terms of $n$ and the moments. We can rewrite $S_n^2$ as\begin{align*} S_n^2=\frac{n\sum_{i=1}^n Z_i^2-(\sum_{i=1}^nZ_i)^2}{n(n-1)}. \end{align*} Squaring leads to\begin{align*} S_n^4=\frac{n^2(\sum_{i=1}^nZ_i^2)^2-2n(\sum_{i=1}^nZ_i^2)(\sum_{i=1}^nZ_i)^2+(\sum_{i=1}^nZ_i)^4}{n^2(n-1)^2} \end{align*} and so\begin{align*} \mathbb{E}(S_n^4)=\frac{n^2\mathbb{E}((\sum_{i=1}^nZ_i^2)^2)-2n\mathbb{E}\left((\sum_{i=1}^nZ_i^2)(\sum_{i=1}^nZ_i)^2 \right)+\mathbb{E}((\sum_{i=1}^n Z_i)^4)}{n^2(n-1)^2}. \end{align*} Since $Z_1,...,Z_n$ are independent, we have that, for distinct $i,j,k$,\begin{align*} \mathbb{E}(Z_iZ_j)=0,\hspace{5mm}\mathbb{E}(Z_i^3Z_j)=0,\hspace{5mm}\mathbb{E}(Z_i^2Z_jZ_k)=0 \end{align*} and\begin{align*} E(Z_i^2Z_j^2)=\mu_2^2=\sigma^4,\hspace{5mm}\mathbb{E}(Z_i^4)=\mu_4. \end{align*} Then, with simple algebraic simplifications, it can be shown that the following holds\begin{align*} \mathbb{E}((\sum_{i=1}^nZ_i^2)^2)&=n\mu_4+n(n-1)\sigma^4,\\ \mathbb{E}\left((\sum_{i=1}^nZ_i^2)(\sum_{i=1}^nZ_i)^2 \right)&=n\mu_4+n(n-1)\sigma^4,\\ \mathbb{E}((\sum_{i=1}^n Z_i)^4)&=n\mu_4+3n(n-1)\sigma^4. \end{align*} Substituting these into the expansion of $\mathbb{E}(S_n^4)$ and simplifying leads to\begin{align*} \mathbb{E}(S_n^4)=\frac{(n-1)\mu_4+(n^2-2n+3)\sigma^4}{n(n-1)} \end{align*} and so\begin{align}\label{var} \mathbb{V}(S_n^2)=\mathbb{E}(S_n^4)-\sigma^4=\frac{(n-1)\mu_4+(n^2-2n+3)\sigma^4}{n(n-1)}-\sigma^4=\frac{\mu_4}{n}-\frac{\sigma^4(n-3)}{n(n-1)}. \end{align}