Consider a situation which is modeled by the Poisson distribution, $P(X_i=k|\theta)=\displaystyle\frac{e^{-\theta}\theta^k}{k!}$. Find:
(a) The MLE of $\theta$.
(b) The MSE of the MLE.
(c) The approximate distribution of the MLE from the Central Limit Theorem.
Okay, so I think I solved (a):
$f(X_1,X_2,…,X_n|\theta)=\displaystyle\frac{e^{-n\theta}\theta^{\Sigma X_i}}{X_1!\cdot…\cdot X_n!}$
so, $\displaystyle\frac{d(lnf)}{\theta}=-n+\displaystyle\frac{\Sigma X_i}{\theta} = 0$
and therefore $\hat\mu=\displaystyle\frac{\Sigma_{i=1}^n X_i}{n}=\boxed{\bar X}$.
If this is correct, how should I go about solving part (b)? I mean, if the MLE that I calculated is unbiased, then isn't the MSE of it just its variance? So, would the answer just be $Var(\bar X)$? Thank you!
Best Answer
Given observed data $\mathbf{x} = (x_1,...,x_n)$ the log-likelihood is:
$$\ell_\mathbf{x}(\theta) = n ( \bar{x} \ln (\theta) - \theta ).$$
This has derivatives:
$$\begin{equation} \begin{aligned} \frac{d \ell_\mathbf{x}}{d \theta}(\theta) &= n \bigg( \frac{\bar{x}}{\theta} - 1 \bigg), \\[6pt] \frac{d^2 \ell_\mathbf{x}}{d \theta^2}(\theta) &= - \frac{n \bar{x}}{\theta^2} <0. \\[6pt] \end{aligned} \end{equation}$$
This shows that the log-likelihood function is concave, so the MLE occurs at the unique critical point given by:
$$0 = \frac{d \ell_\mathbf{x}}{d \theta}(\hat{\theta}) \quad \quad \quad \implies \quad \quad \quad \hat{\theta} = \bar{x}.$$
Since $\mathbb{E}(X) = \theta$ and $\mathbb{V}(X) = \theta$, it follows from the central limit theorem that when $n$ is large, we have the approximate distribution $\hat{\theta} \sim \text{N} ( \theta, \theta/n )$. The estimator is unbiased, and so it has MSE given by:
$$\begin{equation} \begin{aligned} \text{MSE}(\hat{\theta},\theta) = \mathbb{V} (\hat{\theta}) = \mathbb{V} (\bar{X} ) = \frac{\theta}{n}. \end{aligned} \end{equation}$$