I know how how to find the estimation of $\hat{\lambda}$ using the method of moments. I can take the first moment and equate it to the empirical to get,
$E(X) = \frac{1}{\lambda} = \frac{\sum_{i=1}^{n}{x_i}}{n}=\bar{x}$.
This gives $\hat{\lambda}=\frac{1}{\bar{x}}$.
Using the delta method to find $var(\hat{\lambda})$, the result comes out to be $var(\hat{\lambda}) = \frac{\lambda^2}{n}$
My question is how can I obtain a 95% confidence interval for $\lambda$ based on a sample $(x_i, . . . ,x_n)$, when $n$ is large?
My thought is since we are trying to estimate $\lambda$, how can we obtain a confidence interval for $\lambda$? Do I from the confidence interval with $\hat\lambda$? From doing that I get,
$\hat\lambda$ $\pm$ $z_\frac{\alpha}{2}\sqrt{var(\hat\lambda)}$ $\Longrightarrow$ $\hat\lambda$ $\pm$ $z_\frac{\alpha}{2}\sqrt{\frac{\lambda^2}{n}}$ $\Longrightarrow$ $\hat\lambda$ $\pm$ $1.96\sqrt{\frac{\lambda^2}{n}}$
How can the confidence interval of $\lambda$ have $\hat\lambda$ and $\lambda$ in the confidence interval?
Best Answer
There are many different forms of confidence intervals you could use here. In my view, the simplest would be to use the central limit theorem form a probability statement for the difference between the sample mean and the true mean, and then "invert" this to get a corresponding statement for the parameter $\lambda$.
Since the data come from an exponential distribution, the variance is the square of the mean. Thus, applying the central limit theorem, for large $n$, we have the approximate distribution $\bar{X} \sim \text{N}(\mu, \mu^2 /n)$, where $\mu = 1 / \lambda$. This gives us the approximate probability interval:
$$\begin{equation} \begin{aligned} \alpha &\approx \mathbb{P} \Bigg( - z_{\alpha/2} \leqslant \frac{\bar{X} - \mu}{\mu / \sqrt{n}} \leqslant z_{\alpha/2} \Bigg) \\[6pt] &= \mathbb{P} \Bigg( - z_{\alpha/2} \leqslant \sqrt{n} \cdot \frac{1 / \hat{\lambda} - 1/ \lambda }{1 / \lambda} \leqslant z_{\alpha/2} \Bigg) \\[6pt] &= \mathbb{P} \Bigg( - z_{\alpha/2} \leqslant \sqrt{n} \cdot \Big( \frac{\lambda}{\hat{\lambda}} - 1 \Big) \leqslant z_{\alpha/2} \Bigg) \\[6pt] &= \mathbb{P} \Bigg( 1 - \frac{z_{\alpha/2}}{\sqrt{n}} \leqslant \frac{\lambda}{\hat{\lambda}} \leqslant 1 + \frac{z_{\alpha/2}}{\sqrt{n}} \Bigg) \\[6pt] &= \mathbb{P} \Bigg( \hat{\lambda} \Big( 1 - \frac{z_{\alpha/2}}{\sqrt{n}} \Big) \leqslant \lambda \leqslant \hat{\lambda} \Big( 1 + \frac{z_{\alpha/2}}{\sqrt{n}} \Big) \Bigg) \\[6pt] &= \mathbb{P} \Bigg( \frac{1}{\bar{X}} \Big( 1 - \frac{z_{\alpha/2}}{\sqrt{n}} \Big) \leqslant \lambda \leqslant \frac{1}{\bar{X}} \Big( 1 + \frac{z_{\alpha/2}}{\sqrt{n}} \Big) \Bigg). \\[6pt] \end{aligned} \end{equation}$$
Substituting the sample data leads to the confidence interval:
$$\text{CI}_\lambda(1-\alpha) \equiv \Bigg[ \frac{1}{\bar{x}} \Big( 1 - \frac{z_{\alpha/2}}{\sqrt{n}} \Big) , \frac{1}{\bar{x}} \Big( 1 + \frac{z_{\alpha/2}}{\sqrt{n}} \Big) \Bigg].$$
(Note: If $n < z_{\alpha/2}^2$ then the lower bound for this confidence interval will be below zero. You should not use this confidence interval form if this is the case.) Note also from the other answer by BruceET, that you can use the exact distribution of the sample mean (the gamma distribution) to remove the approximation. (The gamma approaches the normal asymptotically, so this wont make much difference when $n$ is large.)