I was studying the method of moments estimation of parameters, and I encountered the following problem.
I have a geometric distribution as following:
$P(X=k) = p(1-p)^{k-1}$, and a sample size of n, all observations are independent
Hence, it can be quite easily shown that
$\hat p = \dfrac{1}{\overline{X}}$, where $\overline{X}$ is the sample mean, this is the MME estimation of p.
Now, if I want to find the approximate normal distribution for $\hat p$, I would have to find out the variance of $\hat p$
Since, $\hat p = \dfrac{n}{\sum\limits_{i=1}^n X_n}$, we see that the bottom follows a negative binomial distribution $NegativeBin(n, p)$.
Now I am stuck at this step, and I can't figure out how to calculate the expression for the PMF and the variance of $\hat p$.
Thank you very much!
Best Answer
Here is a fast Ansatz to find the approximate normal distribution for $\hat p_n$ when $n\to\infty$.
By the CLT, $\sum\limits_{i=1}^nX_i=nm+\sqrt{nv}Z_n$ where $m=\mathbb E(X_1)$, $v=\mathrm{var}(X_1)$ and $Z_n$ converges in distribution to a standard normal random variable. Thus, loosely speaking, $$ \hat p_n=\frac1m\left(1+\frac{\sqrt{v}}{m\sqrt{n}}Z_n\right)^{-1}\approx \frac1m-\frac{\sqrt{v}}{m^2\sqrt{n}}Z_n. $$ In particular, one may hope that $$ \mathbb E(\hat p_n)\approx\frac1m,\quad\mathrm{var}(\hat p_n)\approx\frac{v}{m^4n}. $$ This suggests that $$ \hat p_n=\frac1m+\frac{\sqrt{v}}{m^2\sqrt{n}}V_n, $$ where $V_n$ converges in distribution to a standard normal random variable.