I have a Bernoulli distribution with the parameter $p$ where the $p$ denotes the probability of $X=1$. Then I have five data points $\{X_i\}$ where $3$ of them are $1$ and $2$ of them are $0$. Thus, I have a likelihood of $\mathcal{L}(\{X_i\}, p)=p^3(1-p)^2$.
Given this, I need to compute the Cramer Rao lower bound of $\hat{p}$ that is estimated through MLE.
My lecture note gives the following formula; however, can't understand how to compute the first part inside the $\mathbb{E}$ – $\frac{\partial^{2}}{\partial \theta_{m} \partial \theta_{k}}$.
Any help or hint how to start?
Proposition from Lecture Note
Let $\hat{\theta}$ be the MLE of $\theta$ using datapoints $\left\{X_{i}\right\}_{i=1}^{n}$. Then we have:
$$
\operatorname{Var}(\hat{\theta}) \geq \frac{1}{|\mathcal{I}(\theta)|}
$$
Where the Fisher Information $\mathcal{I}(\theta)$ is a matrix defined as:
$$
\mathcal{I}(\theta)_{m, k}=\mathbb{E}\left(\frac{\partial^{2}}{\partial \theta_{m} \partial \theta_{k}}-\log \mathcal{L}\left(\left\{X_{i}\right\}, \theta\right)\right)
$$
Best Answer
I don't exactly understand whether it is the error in your MathJax, or in the proposition from the lecture notes that causes the confusion. The Fisher Information matrix is supposed to look this way: $$ \mathcal{I}(\theta)_{m, k}=-\mathbb{E}\left(\frac{\partial^{2}}{\partial \theta_{m} \partial \theta_{k}}\log \mathcal{L}\left(\left\{X_{i}\right\}, \theta\right)\right) $$ So to calculate it you have to compute the second derivatives of your log-likelihood function with respect to parameters, just $\hat{p}$ in your case.