Let $\{X_i\}_{i=1}^{n}$ be i.i.d. random variables with distribution $N(\mu, 1)$. Let $p = \mathbb{P}[X_i > 0]$. Find the MLE for $p$ and compute the 95% asymptotic confidence interval.
My attempt:
I know that $\bar{X}$ is the MLE for $\mu$ in normal distribution, and
$$p = \mathbb{P}[X_i > 0] = \mathbb{P}[Z > -\mu] = 1 – \Phi_Z\left(-\mu\right) = 1 – \left[1-\Phi_Z(\mu) \right] = \Phi_Z\left(\mu\right)$$
then, the MLE for $p$ is $\hat{p} = \Phi_Z(\bar{X}).$
The part where you have to calculate the asymptotic confidence interval has me a bit confused since I am working with the estimator of a probability, but I did the following procedure
$$\sqrt{n}(\hat{p}-p) \sim N(0, I(p)^{-1})$$
Then, by calculating Fisher's information, I obtained the following:
$$I(p) = -\mathbb{E}\left[\dfrac{\partial^2}{\partial p^2}\ \log f(X;p)\ \bigg\vert \ p \right] = -\mathbb{E}\left[\dfrac{\partial^2}{\partial p^2}\ \left( \log\left(\dfrac{1}{\sqrt{2\pi}}\right) – \dfrac{(X-p)^2}{2} \right)\ \bigg\vert \ p \right] = -\mathbb{E}[-1] = 1. $$
Then, the 95% asymptotic confidence interval would be as follows:
$$\hat{p} – \dfrac{1.96}{\sqrt{n}}\ \leq \ p \ \leq \ \hat{p} + \dfrac{1.96}{\sqrt{n}}$$
My doubts are as follows:
- Is my procedure for finding the asymptotic confidence interval correct?
- I know that $\bar{X}$ is consistent for $\mu$ in a Normal random sample, and $\Phi_Z(\cdot)$ is a real-valued function continuous in $\mathbb{R}$, so, $\hat{p} = \Phi_Z(\bar{X})$ is consistent. Using this information, is there any other way I can compute the asymptotic confidence interval?
- I have a doubt about consistency: assuming that the sample of random variables is not normal, is it correct to assume that the estimator would still be consistent? My intuition says yes, however, I get confused when I start demonstrating convergence in probability because apparently I am calculating the probability of a probability $\left(i.e.\ \mathbb{P}[\vert \hat{p} – p\vert > \epsilon] \right) $. Can you tell me if my intuition is correct that the estimator would be consistent regardless of the sample distribution, and if possible, give me a hint so I can find the necessary convergence?
Best Answer
I think the way to go here is by the delta method. You've already shown that due to the invariance of the MLE, the MLE estimator is
$$ \hat{p}_{\text{MLE}} = \Phi(\hat{\mu}_{\text{MLE}}) = \Phi (\bar{X}). $$
We can approach the CI similarly (in the sense that we can use what we know about $\bar{X}$ as a starting point). By the CLT, we have $$ \sqrt{n}(\bar{X}-\mu) \implies N(0,1), $$ and so for any continuously differentiable $g$, the delta method tells us that $$ \sqrt{n}(g (\bar{X}) - g(\mu)) \implies N(0, [g'(\mu)]^2 ) $$ Taking $g(\cdot) = \Phi(\cdot)$, we have $g'(x) = \phi(x)$ where $\phi$ is the density function of the standard normal, and so
$$ \sqrt{n}(g (\bar{X}) - g(\mu)) =\sqrt{n}(\Phi (\bar{X}) - \Phi(\mu)) =\sqrt{n}(\hat{p} - p) \implies N(0, \phi^2(\mu) ). $$
Note that $\phi(\bar{X}) \to^p\phi(\mu)$ and so by continuous mapping, $1/\phi(\bar{X}) \to^p 1/\phi(\mu)$ (here, $\to^p$ means 'converges in probability'), and so by Slutsky, $$ \frac{\sqrt{n}(\hat{p}-p)}{\phi(\bar{X})} \implies \frac{1}{\phi(\mu)}N(0, \phi^2(\mu) ) = N(0,1). $$
Therefore, a 95% CI for $p$ is
$$ \hat{p} \pm 1.96 \frac{\phi(\bar{X})}{\sqrt{n}} $$
Since $p \in [0,1]$, you can always clip the CI to never be outside of this range.
Aside: As for your question about consistency, we know that $\bar{X} \to^p \mu$, since $g(x) = \Phi(x)$ is continuous, we can use the continuous mapping theorem to conclude that $\Phi(\bar{X}) \to^p \Phi(\mu)$, i.e. $\hat{p}$ is consistent for $p$.