[Math] How to show that sample mean has the smallest variance

estimationstatistics

Let the population distribution is $N(\mu,1)$.
Sample mean: $\bar{X_n}=\frac{\sum_{i=1}^{n} X_i}{n}$
Then $E(\bar{X_n})=\mu$ and $V(\bar{X_n})=\frac{1}{n}$
It is an unbiased estimator, and as $n \rightarrow \infty$ it converges to $\mu$.
But I want to show that it is the minimun variance estimator, for example,
take another unbiased estimator $Y_n$ and show $V(\bar{X_n}) \le V(Y_n)$.

Best Answer

If you wanted to show only that the sample mean has a smaller variance than every other weighted average of the observations, then this would be an exercise in Lagrange multipliers. But if you want to include all unbiased estimators of $\mu$ based on $X_1,\ldots,X_n$ (for example, the sample median is one such estimator, and is not a weighted average of the observations), then this becomes equivalent to the one-to-one nature of the two-sided Laplace transform.

Observe that the conditional distribution of $(X_1,\ldots,X_n)$ given $\bar X = (X_1+\cdots+X_n)/n$ does not depend on $\mu$. (I could add the details of how to find the conditional distribution if necessary.) In other words, the sample mean $\bar X$ is a sufficient statistic for $\mu$. Therefore, the Rao–Blackwell theorem tells us that any minimum-variance estimator is to be found only among functions of $\bar X$.

Therefore it is enough to show that the only function $g(\bar X)$ of $\bar X$ (where of course, which function $g$ is, is not allowed to depend on $\mu$; i.e. $g(\bar X)$ is actually a statistic) that is an unbiased estimator of $\mu$ is $\bar X$ itself.

The density function of $\bar X$ is $$ x\mapsto \text{constant}\cdot \exp\left(\frac{-1}{2}\cdot\left(\frac{x-\mu}{1/\sqrt{n}}\right)^2\right). $$ In order that the function $g(\bar X)$ be an unbiased estimator of $\mu$, we must $g(\bar X)-\bar X$ being an unbiased estimator of $0$. Let $h(x) = g(x)-x$; then we must have $$ \int_{-\infty}^\infty (\text{same constant})\cdot h(x) \exp\left(\frac{-1}{2}\cdot\left(\frac{x-\mu}{1/\sqrt{n}}\right)^2\right) \, dx = 0 $$ for all values of $\mu$. Hence $$ \text{same constant}\cdot \exp\left(\frac{-n\mu^2}{2}\right) \cdot \int_{-\infty}^\infty \left(h(x) \exp\left(\frac{-n}{2} x^2\right)\right) \exp\left(nx\mu\right) \, dx = 0 $$ regardless of the value of $\mu$. Thus the two-sided Laplace transform of the function $$ x\mapsto h(x)\exp\left( \frac{-nx^2}{2} \right) $$ is $0$ for all values of $\mu$.

Since the two-sided Laplace transform is one-to-one, it can map only one function to the identically zero function.

Related Question