The MLE estimator of the variance of a normal distribution is $\hat \sigma^2
= \frac{1}{N} \sum_{i=1}^{N}(X_i – \bar X)^2$, where $\bar X$ is the sample mean and $X_i \sim^{iid} \mathcal{N}(\mu,\sigma^2)$ . I'm curious because I've seen (e.g. http://www.stat.ufl.edu/~winner/sta6208/allreg.pdf , p.20) that the variance of this estimator is equal to $\frac{2\sigma ^4}{N}$, but I find something different:
Since $\frac{1}{\sigma ^2} \sum_{i=1}^{N}(X_i-\bar X)^2 \sim \chi^2_{N-1}$, we have that
\begin{align}
var\big(\frac{N\hat \sigma^2}{\sigma^2}\big) = var(\chi^2_{N-1}) = 2(N-1) \\
\implies var(\hat \sigma ^2) = \frac{2(N-1) \sigma^4}{N^2}
\end{align}
What am I missing?
Best Answer
The variance of the estimator in the course notes is based on maximum likelihood estimation which typically results in biased estimators. The second variance calculation has a "correction" term that makes the estimator unbiased. You have likely seen this phenomenon with the unbiased estimator for the sample mean, i.e., dividing by $n-1$ instead of $n$.
There are lots of different ways to generate estimators and the resulting estimators will have different properties. One property that many people like in their estimators is for them to be unbiased. However, people are sometimes willing to accept a little bias to reduce variance. If you interested in this topic you might want to look up bias-variance tradeoff.
I noticed several years after my original answer there is a small typo in your derivation that makes a difference: $\frac{(N-1) \hat{\sigma}^2}{\sigma^2} \sim \chi_{n-1}^2$. You will still get a different answer to the one in the notes if you start with this instead of $\frac{N \hat{\sigma}^2}{\sigma^2}$. Starting with this you should find that $$ \textrm{var}\; (\hat{\sigma}^2) = \frac{2\sigma^4}{N-1}. $$