I am trying to calculate Fisher information when $X_1…X_n$ are IID and follow the exponential distribution:
$$f(x)=\frac{1}{\theta}e^{-\frac{x}{\theta}}$$
I use the following formula of Fisher information to confirm that the result is indeed the same as with the other formulas:
$$I(\theta) = E[(\frac{\partial}{\partial \theta }logf(X;\theta))^2]$$
I have calculated that for $\frac{\partial}{\partial \theta }logf(X;\theta)$ we have:
$$\frac {-n}{\theta}+ \frac{\sum_1^nX_i}{\theta^2}$$
Squaring that:
$$\frac{n^2}{\theta^2}+\frac{(\sum_1^n X_i)^2}{\theta^4}-\frac{2n\sum_1^nX_i}{\theta^3}$$
And finally taking the expected value and letting it pass through the constants::
$$\frac{n^2}{\theta^2}+\frac{E(\sum_1^n X_i)^2}{\theta^4}-\frac{2nE(\sum_1^nX_i)}{\theta^3}$$
Recalling that $E(X)= \theta$ and $E(X^2)=2\theta^2$ it seems as though the 2nd and 3rd term cancel out leaving $\frac{n^2}{\theta^2}$, but the correct answer is $\frac{n}{\theta^2}$. Where does this go wrong?
Best Answer
Ok using that parameterisation I agree your likelihood is correct! So method one we differentiate again to get $$ \ell_{\theta \theta} = -\frac{2 \sum x_i }{\theta^3} + \frac{n}{\theta^2} $$ now since $\mathbb{E} \sum_i x_i = n \theta$ we get $$ \begin{align} \mathcal{I} &= - \mathbb{E}\left[ \ell_{\theta \theta} \right] \\ &= 2 \frac{n\theta}{\theta^3} - \frac{n}{\theta^2} \\\ &= \frac{n}{\theta^2} \end{align} $$ Alternatively noting that $$ \begin{align} \mathbb{E}(\sum_i X_i )^2 &= n\mbox{Var}(X_1) + (n\theta)^2 \\ &= n \theta^2 + n^2 \theta^2 \end{align} $$ we have $$ \begin{align} \mathbb{E} \left[ \ell_{\theta} ^2 \right] &= \mathbb{E} \left[ \left(\frac{1}{\theta^2} \sum_i x_i - \frac{n}{\theta} \right)^2\right] \\ &= \frac{1}{\theta^4}\mathbb{E} \left[\left(\sum_i X_i\right)^2 \right]- \frac{2n}{\theta^3}\mathbb{E}\left[\sum X_i \right]+ \frac{n^2}{\theta^2} \\ &= \frac{n\theta^2}{\theta^4} + \frac{n^2 \theta^2}{\theta^4} - \frac{2 n^2 \theta}{\theta^3} + \frac{n^2}{\theta^2} \\ &=\frac{n}{\theta^2}+\frac{2 n^2}{\theta^2} - \frac{2n^2}{\theta^2} \\ &= \frac{n}{\theta^2}. \end{align} $$