Let $X_1, X_2, . . . , X_n$ be iid random variables having pdf
$$f_X(x\mid\theta) =\theta(1 +x)^{−(1+\theta)}I_{(0,\infty)}(x)$$
where $\theta >0$. Give the UMVUE of $\frac{1}{\theta}$ and compute its variance
I have learned about two such methods for obtained UMVUE's:
- Cramer-Rao Lower Bound (CRLB)
- Lehmann-Scheffe Thereom
I am going to attempt this using the former of the two. I must admit that I do not completely understand what is going on here, and I'm basing my attempted solution off of an example problem. I have that $f_X(x\mid\theta)$ is a full one-parameter exponential family with
$h(x)=I_{(0,\infty)}$, $c(\theta)=\theta$, $w(\theta)=-(1+\theta)$, $t(x)=\text{log}(1+x)$
Since $w'(\theta)=1$ is nonzero on $\Theta$, the CRLB result applies. We have
$$\text{log }f_X(x\mid\theta)=\text{log}(\theta)-(1+\theta)\cdot\text{log}(1+x)$$
$$\frac{\partial}{\partial \theta}\text{log }f_X(x\mid\theta)=\frac{1}{\theta}-\text{log}(1+x)$$
$$\frac{\partial^2}{\partial \theta^2}\text{log }f_X(x\mid\theta)=-\frac{1}{\theta^2}$$
so $$I_1(\theta)=-\mathsf E\left(-\frac{1}{\theta^2}\right)=\frac{1}{\theta^2}$$
and the CRLB for unbiased estimators of $\tau(\theta)$ is
$$\frac{[\tau'(\theta)]^2}{n\cdot I _1(\theta)} = \frac{\theta^2}{n}[\tau'(\theta)]^2$$
Since $$\sum_{i=1}^n t(X_i)=\sum_{i=1}^n \text{log}(1+X_i)$$
then any linear function of $\sum_{i=1}^n \text{log}(1+X_i)$, or equivalently, any linear function of $\frac{1}{n}\sum_{i=1}^n \text{log}(1+X_i)$, will attain the CRLB of its expectation, and thus be a UMVUE of its expectation. Since $\mathsf E(\text{log}(1+X))=\frac{1}{\theta}$ we have that the UMVUE of $\frac{1}{\theta}$ is $\frac{1}{n}\sum_{i=1}^n \text{log}(1+X_i)$
For a natural parameterization we can let $\eta=-(1+\theta)\Rightarrow \theta=-(\eta+1)$
Then
$$\mathsf{Var}(\text{log}(1+X))=\frac{d}{d\eta}\left(-\frac{1}{\eta+1}\right)=\frac{1}{(\eta+1)^2}=\frac{1}{\theta^2}$$
Is this a valid solution? Is there a more simple approach? Does this method only work when the $\mathsf E(t(x))$ equals what you're trying to estimate?
Best Answer
Your reasoning is mostly correct.
The joint density of the sample $(X_1,X_2,\ldots,X_n)$ is
\begin{align} f_{\theta}(x_1,x_2,\ldots,x_n)&=\frac{\theta^n}{\left(\prod_{i=1}^n (1+x_i)\right)^{1+\theta}}\mathbf1_{x_1,x_2,\ldots,x_n>0}\qquad,\,\theta>0 \\\\\implies \ln f_{\theta}(x_1,x_2,\ldots,x_n)&=n\ln(\theta)-(1+\theta)\sum_{i=1}^n\ln(1+x_i)+\ln(\mathbf1_{\min_{1\le i\le n} x_i>0}) \\\\\implies\frac{\partial}{\partial \theta}\ln f_{\theta}(x_1,x_2,\ldots,x_n)&=\frac{n}{\theta}-\sum_{i=1}^n\ln(1+x_i) \\\\&=-n\left(\frac{\sum_{i=1}^n\ln(1+x_i)}{n}-\frac{1}{\theta}\right) \end{align}
Thus we have expressed the score function in the form
$$\frac{\partial}{\partial \theta}\ln f_{\theta}(x_1,x_2,\ldots,x_n)=k(\theta)\left(T(x_1,x_2,\ldots,x_n)-\frac{1}{\theta}\right)\tag{1}$$
, which is the equality condition in the Cramér-Rao inequality.
It is not difficult to verify that $$E(T)=\frac{1}{n}\sum_{i=1}^n\underbrace{E(\ln(1+X_i))}_{=1/\theta}=\frac{1}{\theta}\tag{2}$$
From $(1)$ and $(2)$ we can conclude that
These two facts together imply that $T$ is the UMVUE of $1/\theta$.
The second bullet actually tells us that variance of $T$ attains the Cramér-Rao lower bound for $1/\theta$.
Indeed, as you have shown, $$E_{\theta}\left[\frac{\partial^2}{\partial\theta^2}\ln f_{\theta}(X_1)\right]=-\frac{1}{\theta^2}$$
This implies that the information function for the whole sample is $$I(\theta)=-nE_{\theta}\left[\frac{\partial^2}{\partial\theta^2}\ln f_{\theta}(X_1)\right]=\frac{n}{\theta^2}$$
So the Cramér-Rao lower bound for $1/\theta$ and hence the variance of the UMVUE is
$$\operatorname{Var}(T)=\frac{\left[\frac{d}{d\theta}\left(\frac{1}{\theta}\right)\right]^2}{I(\theta)}=\frac{1}{n\theta^2}$$
Here we have exploited a corollary of the Cramér-Rao inequality, which says that for a family of distributions $f$ parametrised by $\theta$ (assuming regularity conditions of CR inequality to hold), if a statistic $T$ is unbiased for $g(\theta)$ for some function $g$ and if it satisfies the condition of equality in the CR inequality, namely $$\frac{\partial}{\partial\theta}\ln f_{\theta}(x)=k(\theta)\left(T(x)-g(\theta)\right)$$, then $T$ must be the UMVUE of $g(\theta)$. So this argument does not work in every problem.
Alternatively, using the Lehmann-Scheffe theorem you could say that $T=\frac{1}{n}\sum_{i=1}^{n} \ln(1+X_i)$ is the UMVUE of $1/\theta$ as it is unbiased for $1/\theta$ and is a complete sufficient statistic for the family of distributions. That $T$ is compete sufficient is clear from the structure of the joint density of the sample in terms of a one-parameter exponential family. But variance of $T$ might be a little tricky to find directly.