Hello I am trying to check if the Method of Moments and Maximum Likelihood Estimators for parameter $\theta$ from a sample with population density $$f(x;\theta) = \frac 2 \theta x e^{\frac {-x^2}{\theta}} $$
for $x \geq 0$, $\theta > 0$ with $\theta$ being unknown.
Taking the first moment of this function I found the Method of Moments estimator to be $\hat{\theta}_1 = \frac{4(\bar{X}^2)}{\pi}$ and solving for the Maximum Likelihood Estimator the Estimator to be $\hat{\theta}_2 = 2\bar Y$ where $Y$ is just square of the Sample $X_i$, i.e. $Y = X_i^2$.
Steps in Solving for Method of Moments:
I took the first moment, i.e.
$M_1 = E[x] = $$\int_0^\infty{\frac 2 \theta x^2 e^{\frac {-x^2}{\theta}}dx}$
Solving this integral with $u$ substitution with $u = \frac{-x}{2}, du = \frac{-1}{2}, v = e^\frac{x^2}{\theta}, dv = -2xe^\frac{-x^2}{\theta}$
$\int_0^\infty{\frac 2 \theta x^2 e^{\frac {-x^2}{\theta}}dx} = [-\frac{xe^\frac{-x^2}{\theta}}{2\theta} – \frac{\sqrt{\pi \theta}}{4}]^\infty_0 = \frac{\sqrt{\pi} {\sqrt{\theta}}}{2}$
So that $E[x] = \bar{x} = \frac{\sqrt{\pi} {\sqrt{\theta}}}{2}$ gives the Method of Moments Estimator $\hat{\theta_1} = \frac{4(\bar{X}^2)}{\pi}$
Steps in Solving for Maximum Likelihood:
$lnL(\theta)=(\prod_{i=1}^n\frac 2 \theta x e^{\frac {-x^2}{\theta}}) = -n ln((2\theta)) + \sum_{i=1}^nx_i – \frac {1} {\theta} \sum_{i=1}^nx^2_i$
$\frac {dlnL(\theta)}{d\theta} = \frac{-n}{2\theta} + \frac{1}{\theta^2} \sum_{i=1}^nx^2_i$
Setting $\frac {dL(\theta)}{d\theta} = 0$, I found the Maximum Likelihood Estimator $\hat{\theta_2}$ to be $\hat{\theta_2} = \frac{2\sum_{i=1}^nx^2_i}{n}$ , so that if $Y = X_i^2$ then $\hat{\theta_2} = 2\bar{Y}$.
I am trying to check if these estimators for $\theta$ from this density function are unbiased and/or consistent but am lost on how to go about doing so, any help would be much appreciated.
Best Answer
Your calculation of $\hat \theta_2$ is still not right. You should have $$\mathcal L(\theta \mid \boldsymbol x) = 2^n \theta^{-n} \left ( \prod_{i=1}^n x_i \right) \exp \left( -\frac{1}{\theta} \sum_{i=1}^n x_i^2 \right) \propto \theta^{-n} \exp \left( - \frac{n \overline{x^2}}{\theta} \right),$$ thus your log-likelihood is $$\ell (\theta \mid \boldsymbol x) = -n \log \theta - \frac{n \overline{x^2}}{\theta},$$ and locating the critical points gives $$0 = \frac{\partial \ell}{\partial \theta} = -\frac{n}{\theta} + \frac{n \overline{x^2}}{\theta^2} = n \left( \frac{\overline{x^2} - \theta}{\theta^2} \right),$$ hence $\hat \theta_2 = \overline{x^2} = \frac{1}{n} \sum_{i=1}^n x_i^2.$ There is no additional factor of $2$. Notice how I remove all factors of $\mathcal L$ that are not functions of $\theta$, which simplifies all subsequent calculations (and avoids the computational error you made with the additional factor of $2$).
To compute the bias of these estimators, it suffices to use the basic properties: First, observe $$\operatorname{E}[X] = \frac{\sqrt{\pi \theta}}{2}$$ as you wrote. You may verify that $$\operatorname{E}[X^2] = \theta.$$ Now we see that $$\operatorname{E}[\hat\theta_2] = \operatorname{E}\left[\frac{1}{n} \sum_{i=1}^n X_i^2\right] = \frac{1}{n} \sum_{i=1}^n \operatorname{E}[X_i^2] = \frac{1}{n} \cdot n \theta = \theta,$$ so $\hat\theta_2$ is unbiased. This is the most immediately obvious calculation which is why we started with it. As for $\hat \theta_1$, we must be careful to write $$\operatorname{E}[\hat\theta_1] = \frac{4}{\pi} \operatorname{E}\,\left[\left(\frac{1}{n} \sum_{i=1}^n X_i\right)^2\right] = \frac{4}{\pi n^2} \sum_{i=1}^n \sum_{j=1}^n \operatorname{E}[X_i X_j].$$ Note we have not yet used independence of the sample, only the linearity of expectation. When $i \ne j$, $X_i$ and $X_j$ are independent, and $$\operatorname{E}[X_i X_j] = \operatorname{E}[X_i]\operatorname{E}[X_j] = \frac{\pi}{4}\theta.$$ But when $i = j$, this is not the case and we have $$\operatorname{E}[X_i X_j] = \operatorname{E}[X_i^2] = \theta.$$ Since the first case occurs $n(n-1)$ times in the double sum, and the second occurs $n$ times, we get $$\operatorname{E}[\hat\theta_1] = \frac{4}{\pi n^2} \left( n(n-1) \frac{\pi}{4}\theta + n \theta \right) =\left( 1 + \frac{4 - \pi}{\pi n} \right) \theta .$$ This of course proves $\hat\theta_1$ is biased. Is it asymptotically biased or unbiased?
As for the consistency of the estimator, you must show using similar methods that the variance decreases with increasing sample size $n$. To do this, you must compute $\operatorname{E}[\hat\theta_1^2]$ and $\operatorname{E}[\hat\theta_2^2]$. This is left as an exercise.
The second moment of $\hat \theta_1$ is $$\operatorname{E}[\hat\theta_1^2] = \frac{4^2}{\pi^2} \operatorname{E}\,\left[\left(\frac{1}{n}\sum_{i=1}^n X_i\right)^4\right].$$ Use the same technique as for the first moment: $$\left(\sum_{i=1}^n X_i\right)^4 = \sum_{g=1}^n \sum_{h=1}^n \sum_{i=1}^n \sum_{j=1}^n X_g X_h X_i X_j.$$ How many of these summands correspond to all distinct indices? How many correspond to exactly two equal? How many correspond to two equal pairs? How many correspond to exactly three equal? How many correspond to all four equal? What is the expectation of a general term in each case?