UMVUE – Explanation of Why an UMVUE Doesn’t Necessarily Achieve the Cramer-Rao Lower Bound

estimationexponential-familyfisher informationmathematical-statisticsumvue

I'm studying uniformly minimum variance unbiased estimator(UMVUE). I have seen question on this site asking why the UMVUE doesn't achieve the CRLB(Cramer Rao lower bound), and all of the answers have been of the type where they give examples of some random distribution and estimator which is UMVUE but does not achieve CRLB.

My question is different.(At least I think so).

Is there a general explanation of why an UMVUE doesn't necessarily have to achieve the CRLB? Either intuitive or logical explanations would be appreciated

Best Answer

While the CRLB is an inequality, and there is in general no reason for CRLB to hold with equality, it is in fact possible to say something about that possibility. A good book of theoretical statistics that does so, is Young and Smith: Essentials of Statistical Inference. I will try to review here what they do (around page 125.)

Let $W(X)$ be an unbiased estimator for the scalar parameter $\theta$. Then the CRLB is $\DeclareMathOperator{\V}{\mathbb{V}} \V W(X) \ge \frac1{i(\theta)}$, where $i(\theta)$ is the Fisher information. The proof of this uses the correlation inequality (a version of the Cauchy-Schwarz inequality) $$ \DeclareMathOperator{\C}{\mathbb{C}} \C (Y,Z) \le \V(Y) \V(Z) $$ with $Y=W(X), Z=\frac{\partial}{\partial \theta} \log f(X; \theta)$. Equality is only possible if $\DeclareMathOperator{\Cor}{\mathbb{Cor}} \Cor(Y,Z)=\pm 1$, which only is possible if $Y$ and $Z$ are proportional to each other (as functions of $X$ for each $\theta$.)

So it is necessary that $$ \frac{\partial}{\partial \theta} \log f(X; \theta) = a(\theta) \left( W(X)-\theta\right) $$ for some functions $a(\theta)$. Now on integration $$ \log f(X;\theta) = A(\theta) W(X) + B(\theta) + C(X) $$ for some functions $A, B, C$. This says that $F(X;\theta)$ is an exponential family model.

Conclusion: For equality in the CRLB to be possible, the model must be an exponential family. Note that this is necessary, but not suficient, the argument above do give not only an exponential family, but is is also parametrized such that $\DeclareMathOperator{\E}{\mathbb{E}} \E W(X)=\theta$.