So I am doing some self studying in preparation for the Statistical Inference course next semester. I stumbled upon this question which is:
If you have a random sample of size $n$ from a geometric distribution with a mean of $1/p,$ find the Cramer-Rao lower bound for the variance of an unbiased estimator of $p.$
Here is my attempt so far
We know that if $T$ is an unbiased estimator of $\tau(\theta)$, if the Cramer-Rao lower bound, based on a random sample is:
$$\operatorname{Var}(T) \ge \frac{[\tau'(\theta)]^2}{n\operatorname E \left[\left( \frac{\partial}{\partial \theta} \ln f(X;\theta)\right)^2\right]}$$
Solving for $\tau(\theta)$
$$\tau(\theta) = p$$
$$\tau'(\theta) = 1$$
$$[\tau'(\theta)]^2 = 1$$
Solving for $f(x;\theta)$
$$f(x;\theta) = p(1-p)^{x-1}$$
$$\ln f(x;\theta) = \ln p + (x-1) \ln(1-p) = \ln p + x\ln(1-p) – \ln(1-p)$$
$$\frac{\partial \ln f}{\partial p } = \frac{1}{p} – \frac{x-1}{1-p}$$
I don't know what to do next. Please help!
Best Answer
First you would have to consider the square of your last expression, and evaluate that expression for $x=X$ (that is, evaluate at the r.v. $X$). Finally, you would have to take expectation, using the fact that $X$ has the mentioned geometric distribution. That is:
$$E\left[\left(\frac1p-\frac{X-1}{1-p}\right)^2\right]=E\left[\left(\frac{1-pX}{p(1-p)}\right)^2\right]=\frac1{p^2(1-p)^2}E\left(1-2pX+p^2X^2\right)=\cdots$$ for which you will have to know which is $E(X^2)$ (you already know that $E(X)=\tfrac1p$).
Nevertheless, there's an equivalent expression that you get by changing the denominator by $$-nE\left(\frac{\partial^2}{\partial \theta^2}\ln f(X;\theta)\right),$$ provided this second derivative exists. This alternative formula gives simpler calculations many times, not having to take expectation of any square (and I find it works fine in this case).