Solved – Unbiased estimator for variance or Maximum Likelihood Estimator

maximum likelihoodvariance

I know that $\frac{1}{n-1}\sum_{i=1}^{n}(X-\bar{X_{n}})^{2}$ is an unbiased estimator for the variance.

I thought of such an estimator to be useful when it is not know from which distribution the data at hand are coming from.

Now given I have a data set and I do know which is the underlying distributions of the observations. Shouldn't I prefer a Maximum Likelihood Estimator for the variance even if it is biased since it obviously makes use of the underlying distribution of the data over the general variance estimator cited above, which is not for any specific distribution?

Thanks

Best Answer

I think the answer is generally yes. If you know more about a distribution then you should use that information. For some distributions this will make very little difference, but for other it could be considerable.

As an example, consider the poisson distribution. In this case the mean and the variance are both equal to the parameter $\lambda$ and the ML estimate of $\lambda$ is the sample mean.

The charts below show 100 simulations of estimating the variance by taking the mean or the sample variance. The histogram labelled X1 is the using sample mean, and X2 is using the sample variance. As you can see, both are unbiased but the mean is a much better estimate of $\lambda$ and hence a better estimate of he variance.

enter image description here

The R code for the above is here:

library(ggplot2)
library(reshape2)
testpois = function(){
  X = rpois(100, 4)
  mu = mean(X)
  v = var(X)
  return(c(mu, v))
}

P = data.frame(t(replicate(100, testpois())))
P = melt(P)

ggplot(P, aes(x=value)) + geom_histogram(binwidth=.1, colour="black", fill="white") +
  geom_vline(aes(xintercept=mean(value, na.rm=T)),   # Ignore NA values for mean
             color="red", linetype="dashed", size=1) + facet_grid(variable~.)

As to the question of bias, I wouldn't worry too much about your estimator being biased (in the example above it isn't, but that is just luck). If unbiasedness is important to you you can always use Jackknife to try remove the bias.

Related Question