Solved – what does it mean by more “efficient” estimator

hypothesis testinginferencemathematical-statistics

When comparing two estimators, say $T_1$ and $T_2$, what does it mean by saying $T_1$ is more efficient than $T_2$?

Could someone give an easy but very concrete example?

Also I have another question about relative efficiency:
https://en.wikipedia.org/wiki/Efficient_estimator

It says in the above Wikipedia article that:
If $T_1$ and $T_2$ are estimators for the parameter $\theta$, then $T_1$ is said to dominate $T_2$ if:

1) its mean square is smaller for at least some value of $\theta$

2) the MSE does not exceed that of $T_2$ for any value of $\theta$

Formally $T_1$ dominates $T_2$ if

$E(T_1-\theta)^2 \leq E(T_2-\theta)^2$

I have two questions:

1) If we don't know $\theta $, then how can we show one is smaller than the other in the above inequality.

2) Also I thought there is a SINGLE "true" value of the parameter $\theta$, is it correct? then what does it mean by saying "for SOME value of $\theta$" in the above statement in Wikipedia? If there is only ONE, why does it say "for SOME" value of $\theta$?

Best Answer

I am just wondering, when comparing two estimator says T1 and T2, what does it mean by saying T1 is more efficient than T2

https://en.wikipedia.org/wiki/Efficiency_(statistics)

For an unbiased estimator, efficiency is the precision of the estimator (reciprocal of the variance) divided by the upper bound of the precision (which is the Fisher information). Equivalently, it's the lower bound on the variance (the Cramer-Rao bound) divided by the variance of the estimator.

The relative efficiency of two unbiased estimators is the ratio of their precisions (the bound cancelling out)

When you're dealing with biased estimators, relative efficiency is defined in terms of the ratio of MSE.

Could someone give an easy but very concrete example.

Compare the sample mean ($\bar{x}$) and sample median ($\tilde{x}$) when trying to estimate $\mu$ at the normal.

They're both unbiased so we need the variance of each. The variance of the median for odd sample sizes can be written down from the variance of the $k$th order statistic but involves the cdf of the normal. In large samples $\frac{n}{\sigma^2}\text{ Var}(\tilde{x})$ approaches the asymptotic value reasonably quickly, so people tend to focus on the asymptotic relative efficiency.

The asymptotic relative efficiency of median vs mean as an estimator of $\mu$ at the normal is the ratio of variance of the mean to the (asymptotic) variance of the median when the sample is drawn from a normal population.

This is $\frac{\sigma^2/n}{2\pi \sigma^2/(4 n)} = 2/\pi\approx 0.64$

There's another example discussed here: Relative efficiency: mean deviation vs standard deviation

If we don't know θ, then how can we show one is smaller than the other in the above inequality.

Generally the MSE's will be some function of $\theta$ and $n$ (though they may be independent of $\theta$). So at any given $\theta$ you can compute their relative size.

Also I thought there is a SINGLE "true" value of the parameter θ, is it correct?

Yes, at least in the usual situations we'd be doing this in and assuming a frequentist framework.

then what does it mean by saying "for SOME value of θ" in the above statement [...] if there is only ONE, why it says "for SOME" value of θ

When you are comparing estimators you want ones that do well for every value of $\theta$. If you don't know what $\theta$ is (if you did, you wouldn't have to bother with estimators), it would be good if it worked well for whatever value you have.

Related Question