I'm having a problem when calculating the variance of the following estimator:
$\hat\theta=\frac{1}{N}\sum_{n=1}^{N}D_n$ with $D_1….D_N$ independent random variables.
In order to calculate the variance of this estimator, I follow the following procedure:
$\sigma_{\hat\theta}^{2}=E[(\hat\theta – E[\hat\theta])^2]=E[\hat\theta^2]+E[E[\hat\theta]^2]-2E[\hat\theta]E[E[\hat\theta]]$
Knowing that $f(D)=\frac{1}{\theta}e^{-\frac{1}{\theta}D}$ and therefore, $E[D]= E[\hat\theta]=\theta$.
From this point, I get that $\sigma_{\hat\theta}^{2} = E[\hat\theta^2] +\theta^2-2\theta^2$
So, in order to obtain $E[\hat\theta^2] = \frac{1}{N^2}\sum_{n=1}^{N}\sum_{m=1}^{N}E[D_nD_m]$ that will be $E[D^2]$ for $n=m$ and $E[D_n]E[D_m]=\theta^2$ for $n \neq m$. At this point, when calculating $E[D^2]$ from its pdf I get that it is equal to 0, so I would obtain:
$E[\hat\theta^2] = \frac{1}{N^2}(N^2-N)\theta^2$ taking into account that there are $(N^2-N)$ values that satisfy the condition $n \neq m$ and therefore,
$\sigma_{\hat\theta}^{2} = \frac{1}{N^2}(N^2-N)\theta^2 +\theta^2-2\theta^2=-\frac{\theta^2}{N}$, which I know it is not ok since it is negative. Why could this be happening?
Any help is appreciated!
Best Answer
Your $D$ just has an exponenital distribution $f(d)=\frac{1}{\theta}e^{-\frac{1}{\theta}d}$ wtih $E(D)=\theta$ and $Var(D)=\theta^2$.
Now your $\hat{\theta}$ is just the mean, by i.i.d
$Var(\hat\theta)=Var(\frac{1}{N}\sum_{n=1}^{N}D_n)=(\frac{1}{N})^2\theta^2+(\frac{1}{N})^2\theta^2+...+(\frac{1}{N})^2\theta^2$, totally there are $N$ such terms.
Therefore,the variance is $ \frac{1}{N}\theta^2$