Consider $X_1, X_2, \ldots, X_n$ i.i.d $N(\mu, \sigma^2),$ where both parameters are unknown, and consider estimation of $\sigma^2.$
Consider the MLE for $\sigma^2.$ We know it is,
$$\hat{\sigma^2_n} = \frac{1}{n} \sum_{i=1}^n \left(X_i-\bar{X} \right)^2.$$
I have shown $\hat{\sigma^2_n}$ is biased but consistent for $\sigma^2.$ Now, I want to add a concrete example to visualize the property of consistency in $\hat{\sigma^2_n}$ and ran into the following hurdle.
I considered $X_1, X_2, \ldots, X_n$ i.i.d $N(2, 3^2)$. Took random samples of size $n$. Have a plot that shows how as $n$ increases, $\hat{\sigma}^2_n \to 9$ in probability. This makes sense since $\hat{\sigma^2_n}$ is consistent.
But then I did the following:
I repeatedly ($100,000$ times) took samples of size $n=20$, calculated $\hat{\sigma}^2_n$, and visualized with histogram to get:
I repeatedly ($100,000$ times) took samples of size $n=10,000$, calculated $\hat{\sigma}^2_n$, and visualized with histogram to get:
My question is, are these histograms showing that $\hat{\sigma^2_n}$ is asymptotically unbiased (not always true for all consistent estimators but true for this one)? Or is it showing that $\hat{\sigma^2_n}$ is also an efficient estimator (what is the Cramer Rao Lower Bound when $\mu$ is also unknown, and is the asymptotic variance of $\hat{\sigma^2_n}$ equal to it)?
Best Answer
Your histograms seem to suggest that for very large $n$ the distribution concentrates around $9$ which is $\sigma^2$.
You might add a vertical line to each of the means of the variance estimates, with the first somewhere presumably near $9.47$ and the second much nearer $9$ to suggest asymptotically unbiasness. If this is difficult to visualise (the first line will be closer to $10$ than the second) then repeat the first with perhaps $n=10$ and a vertical line near $10$