Problem description: Find the Fisher information of the Rayleigh distribution. I was satisfied with my solution until I saw that it disagreed with the solution obtained in one of the problem sets from Princeton. http://www.princeton.edu/~cuff/ele530/files/hw4_sn.pdf p2. I calculate the Fisher information by the following Thm: $$I(\theta) = -E(\frac{\partial^2 \log L}{\partial \theta^2}) $$ where L is the likelihood function of the pdf. The princeton problem set uses another argument I am not familiar with, where they obtain $I(\theta) = \frac{n}{\theta^2}$. I will show my calculations below, maybe someone can spot the error(if there is one). $$\log L = \prod \log f(x_{i})$$ where $X_{i}$'s are iid Rayleigh distributed for $i=1,2,…n$. I end up with $$\log L = \sum_{i}(\log x_{i} -\frac{2}{\theta} – \frac{x_{i}^2}{2\theta^2}) $$ Then I take the partial derivative with respect to $\theta$ two times and obtain $$\frac{\partial^2 \log L}{\partial \theta^2} = \frac{2n\theta^2-3\sum_{i}x_{i}^2}{\theta^4} $$ So $I(\theta) = -E(\frac{2n\theta^2-3\sum_{i}x_{i}^2}{\theta^4} )$. This is where I am confused, should this expected value be evauated with respect to the $x_{i}^2$'s or $\theta$?
[Math] Fisher information of the Rayleigh distribution
expectationprobability distributions
Related Solutions
So, you have $X$ ~ Binomial($n$, $p$), with pmf $f(x)$:
You seek the Fisher Information on parameter $p$. Here is a quick check using mathStatica's FisherInformation function:
which is what you got :)
Ok using that parameterisation I agree your likelihood is correct! So method one we differentiate again to get $$ \ell_{\theta \theta} = -\frac{2 \sum x_i }{\theta^3} + \frac{n}{\theta^2} $$ now since $\mathbb{E} \sum_i x_i = n \theta$ we get $$ \begin{align} \mathcal{I} &= - \mathbb{E}\left[ \ell_{\theta \theta} \right] \\ &= 2 \frac{n\theta}{\theta^3} - \frac{n}{\theta^2} \\\ &= \frac{n}{\theta^2} \end{align} $$ Alternatively noting that $$ \begin{align} \mathbb{E}(\sum_i X_i )^2 &= n\mbox{Var}(X_1) + (n\theta)^2 \\ &= n \theta^2 + n^2 \theta^2 \end{align} $$ we have $$ \begin{align} \mathbb{E} \left[ \ell_{\theta} ^2 \right] &= \mathbb{E} \left[ \left(\frac{1}{\theta^2} \sum_i x_i - \frac{n}{\theta} \right)^2\right] \\ &= \frac{1}{\theta^4}\mathbb{E} \left[\left(\sum_i X_i\right)^2 \right]- \frac{2n}{\theta^3}\mathbb{E}\left[\sum X_i \right]+ \frac{n^2}{\theta^2} \\ &= \frac{n\theta^2}{\theta^4} + \frac{n^2 \theta^2}{\theta^4} - \frac{2 n^2 \theta}{\theta^3} + \frac{n^2}{\theta^2} \\ &=\frac{n}{\theta^2}+\frac{2 n^2}{\theta^2} - \frac{2n^2}{\theta^2} \\ &= \frac{n}{\theta^2}. \end{align} $$
Best Answer
Continuing the above discussion, I find the following when computing $-E(\frac{2n\theta^2-3\sum_{i}x_{i}^2}{\theta^4} )$. $$-E(\frac{2n\theta^2-3\sum_{i}x_{i}^2}{\theta^4}) = -E(\frac{2n\theta^2+3\sum_{i}-x_{i}^2}{\theta^4})$$ now since I found that $$E(T(x)) = E(-x_{j}^2) = -2\theta^2 $$ for arbitrary $x_{j}$. $$E(\frac{2n\theta^2+3\sum_{i}-x_{i}^2}{\theta^4}) = \frac{2n}{\theta^2} + \frac{3nE(T(x))}{\theta^4} = \frac{-4n}{\theta^2} $$ since E is linear and $x_{i}$'s are iid, hence $$-E(\frac{2n\theta^2+3\sum_{i}-x_{i}^2}{\theta^4}) = \frac{4n}{\theta^2}$$