Suppose we have $T_i,i=1..n$ i.i.d. with unknown distribution and we want to estimate $E[T]$. Note that in this setting we are not estimating E[T] as a parameter of a parameter-dependent family of distributions, therefore it is difficult to attach a meaning to a likelyhood function:
$L[E[T]]=P(T_i|E[T])$
as it would be done when for example estimating $\mu$ from maximum likelyhood, knowing that the underlying distribution is Gaussian.
Again what we would probably do is computing $\overline{T}=\frac{1}{n}\sum_i T_i$. And we would be sure that the estimator would be consistent and its variance would go to zero. For example we have trivially:
$E[\overline{T}]=E[T_i]$
and:
$\sigma^2[\overline{T}]=O(1/n)$
supposing that $T_i$ has finite variance.
Here is the question: can we prove that $\overline{T}$ is optimal in some way? To me conceptes from MLE estimators or sufficient statistics are a bit difficult to apply, since $E[T]$ is not a parameter of the distribution but maybe I am missing something? Can we "derive" the sample mean estimator to be optimal according to some criterion in the general case and without making assumptions on the underlying distributions ?
Best Answer
Though the sample mean is an unbiased estimator of the unknown population mean, it cannot be optimal in general. Take the case of the log-normal distribution. The maximum likelihood estimator of the mean on the original scale is a function of the sample mean and sample variance both computed on the log scale. This prevents outliers from ruining either the mean or SD. There is a relationship of this problem with that fact that an accurate nonparametric confidence interval for the population mean does not exist. When one wants to have a measure that 'works' on all continuous distributions, one has to use an estimator that aligns with nonparametrics, such as the sample median. BY doing so one pays a high efficiency price if the data are Gaussian, since the sample median in that case is an inefficient estimator of the population mean or median.