Given,
$$f_\theta(x) = \theta x^{\theta-1}, x \in [0,1], \theta >0$$
$$\hat\theta_\text{MLE} = \frac{-1}{\frac{1}{n} \sum_{i=1}^n \log(x_i)} $$
$$\operatorname{Var}(\hat\theta_\text{MLE}) = E_\theta(\hat\theta_\text{MLE}^2) – E_\theta(\hat\theta_\text{MLE})^2$$
How do I find the expectations, $E_\theta(\hat\theta_\text{MLE}^2)$ and $E_\theta(\hat\theta_\text{MLE})$, given the presence of both summation and log in the denominator?
Best Answer
Since $$f_X(x) = \theta x^{\theta-1},$$ then $Y = -\log X$ has density $$f_Y(y) = f_X(e^{-y}) e^y = \theta e^{-(\theta-1)y} e^y = \theta e^{-\theta y}, \quad y > 0.$$ Consequently $Y \sim \operatorname{Exponential}(\theta)$, and $$W = -\sum_{i=1}^n \log X_i \sim \operatorname{Gamma}(n,\theta)$$ with density $$f_W(w) = \frac{\theta^n w^{n-1} e^{-\theta w}}{\Gamma(n)}.$$ It follows that $\hat\theta_{\text{MLE}} = n/W$ has density $$f_{\hat \theta}(z) = f_W(n/z)\left|\frac{d}{dz}\left[\frac{n}{z}\right]\right| = \frac{n}{z^2} \frac{\theta^n (n/z)^{n-1} e^{-\theta n/z}}{\Gamma(n)} = \frac{(n\theta)^n e^{-(n\theta)/z}}{z^{n+1} \Gamma(n)}$$ which implies $$\hat \theta_{\text{MLE}} \sim \operatorname{InverseGamma}(n,n\theta).$$ I leave as an exercise to the reader to show that $$\operatorname{Var}[\hat \theta_{\text{MLE}}] = \frac{(n\theta)^2}{(n-1)^2 (n-2)}, \quad n > 2.$$