Suppose we have a random sample $(X_{1},…..,X_{n})$, where $X_{i}$ follows an Exponential Distribution with parameter $\lambda$, hence:
$F(x) = 1 – exp(-\lambda x)$
$\mathbb{E}(X_{i}) = \dfrac{1}{\lambda}$
$ Var(X_{i}) =\dfrac{1}{\lambda^{2}}$
I know that the MLE estimator $\hat{\lambda} = \dfrac{n}{\sum_{i=1}^{n}X_{i}}$, asymptotically follows a normal distribution, but I'm interested in his variance.
So, since $\sqrt n (\hat{\lambda} – \lambda)\stackrel{D}{\rightarrow} \mathcal{N}(0, \sigma^{2}) $
I was thinking about dealing with:
\begin{align}
\sigma^{2} &= Var\left[\sqrt n (\hat{\lambda} – \lambda)\right]\\
&= n Var (\hat{\lambda})\\
&= n Var \left[ \dfrac{n}{\sum_{i=1}^{n}X_{i}} \right] \\
&= n^{3} Var \left[ \dfrac{1}{\sum_{i=1}^{n}X_{i}}\right]
\end{align}
But I don't know how to proceed from here. I think the sum of exponentials follows a gamma with parameters $(n,\lambda)$, and that $\dfrac{1}{\sum_{i=1}^{n}X_{i}}$ then follows an inverse gamma?
Best Answer
Yes you are almost there.
You can calculate the variance of $\hat{\lambda}=\frac{n}{Y}$ remembering that
$$Y=\Sigma_i X_i \sim \text{Inverse Gamma}$$
and thus you immediately solve the problem as its variance is a known parameter
...but you can also solve the problem using the integration, and calculating
$$\mathbb{V}\Bigg[\frac{1}{Y}\Bigg]=\mathbb{E}\Bigg[\frac{1}{Y^2}\Bigg]-\mathbb{E}^2\Bigg[\frac{1}{Y}\Bigg]$$
It is not difficult:
$$\mathbb{E}\Bigg[\frac{1}{Y}\Bigg]=\int_0^{\infty}\frac{1}{y}\frac{\lambda^n}{\Gamma(n)}y^{n-1}e^{-\lambda y}dy=\frac{\lambda}{n-1}\underbrace{\int_0^{\infty}\frac{\lambda^{n-1}}{\Gamma(n-1)}y^{(n-1)-1}e^{-\lambda y}dy}_{=1}=\frac{\lambda}{n-1}$$
and similarly for the second simple moment
Anyway this is not the asymptotic variance but it is the exact variance. To calculate the asymptotic variance you can use Delta Method
After simple calculations you will find that the asymptotic variance is $\frac{\lambda^2}{n}$ while the exact one is $\lambda^2\frac{n^2}{(n-1)^2(n-2)}$