This question is from George Casella statistical inference textbook 10.35 (b).
Let $X_1,…,X_n$ be a random sample from a $n(\mu,\sigma^2)$ population. If $\sigma^2$ is unknown and $\mu$ is known, find a Wald statistic for testing $H_0: \sigma =\sigma_0$.
My attempt: I refer to textbook page 493. First, I need to find the MLE of $\sigma^2$. I find it and is the same as the solution. The log likelihood is
$$-\frac{n}{2}log(2\pi\sigma^2)-\frac{1}{2\sigma^2}\Sigma(x_i-\mu)^2$$. Then taking derivatives with $\sigma^2$, I get the MLE of $\sigma^2$ is $\frac{\Sigma(x_i-\mu)^2}{n}$. Then I got problems. According to page 473, next I need to get the approximate variance of the estimator. First, I need to get the observed information number. Just taking second derivatives and then adding a minus sign).
My frist derivative w.r.t $\sigma$ is:
$$-\frac{n}{\sigma}+\frac{\Sigma(x_i-\mu)^2}{\sigma^3}$$
My second derivative w.r.t $\sigma$ is:
$$\frac{n}{\sigma^2}-\frac{3\Sigma(x_i-\mu)^2}{\sigma^4}$$
Add a minus sign:$$-\frac{n}{\sigma^2}+\frac{3\Sigma(x_i-\mu)^2}{\sigma^4}$$
Now plug in $\sigma=\sqrt{\frac{\Sigma(x_i-\mu)^2}{n}}$, I got the observed information number is $\frac{2n^2}{\Sigma(x_i-\mu)^2}$. Hence, according to page 473 formula, my variance of the MLE of $\sigma$ is the reciprocal, that is $\frac{\Sigma(x_i-\mu)^2}{2n^2}$. Hence compared with the. solution, our numerator is the same. But my denominator is $\sqrt{\frac{\Sigma(x_i-\mu)^2}{2n^2}}$, different from the solution. What step did I get wrong?
The solution is here:
One comment below said I cannot take derivatives r.p.t $\sigma$. I don't know why. I think it's okay for me to do this, because my goal is to get the approximate variance of $\sigma^2$.
But if I choose to take derivatives r.p.t $\sigma^2$ as suggested, then my first derivative of log likelihood is
$$-\frac{n}{2\sigma^2}+\frac{\Sigma(x_i-\mu)^2}{2(\sigma^2)^2}$$. My second derivative is
$$\frac{n}{2(\sigma^2)^2}-\frac{\Sigma(x_i-\mu)^2}{(\sigma^2)^3}$$ Then add a minus sign is:
$$-\frac{n}{2(\sigma^2)^2}+\frac{\Sigma(x_i-\mu)^2}{(\sigma^2)^3}$$
Now plug in MLE $\sigma^2=\frac{\Sigma(x_i-\mu)^2}{n}$, I got the observed information number is $\frac{n^3}{2(\Sigma(x_i-\mu)^2)^2}$. Now it is the same as the solution.
But I am still confused why my previous method to take derivatives r.p.t $\sigma$ failed. My idea is if I take derivatives r.p.t $\sigma$ directly, then I don't need to use delta method to get the variance of $\sigma$ from the variance of of $\sigma^2$
Also, I am stuck how to get the next. According to the solution, now I should use delta method to get the variance of $\sigma$.
My preferred version of delta method is if $W_n \sim AN(a, b_n)$, where $b_n$ goes to 0, and g is differentiable with $g'(a)$ not 0, then $g(W_n) \sim AN(g(a), [g'(a)]^2 b_n)$. (AN denotes approximate normal).
So my $W_n$ is $\frac{\Sigma(x_i-\mu)^2}{n}$, my $a$ is $\sigma^2$, my $b_n$ is $\frac{2(\Sigma(x_i-\mu)^2)^2}{n^3}$. My g is $\sqrt{}$. Hence, my approximated variance of $\sigma$ is
$$[g'(a)]^2 b_n=(\frac{1}{2\sqrt{a}})^2 b_n=\frac{1}{4a} b_n=\frac{1}{4\sigma^2} \frac{2(\Sigma(x_i-\mu)^2)^2}{n^3}$$.
It's weird. My approximated variance of $\sigma$ contains $\sigma^2$. Something in my attempt is wrong here.
Best Answer
It's official. I disagree with the solution you have provided. Let me share my perspective on this problem.
The Fisher information $\mathcal{I}(\sigma^2)$ can be expressed as $$\mathcal{I}(\sigma^2)=\mathbb{E}\Bigg(-\frac{\partial ^2l(X_1,\ldots,X_n;\sigma^2)}{\partial (\sigma^2)^2}\Bigg|\sigma^2\Bigg)$$ where $l(x_1,,\ldots,x_n;\sigma^2)$ is your "log$-$likelihood" $$l(x_1,,\ldots,x_n;\sigma^2)=-\frac{n}{2}\ln(2\pi \sigma^2)-\frac{1}{2\sigma^2}\sum_{i=1}^n(x_i-\mu)^2$$ You accurately found $-\frac{\partial ^2l(X_1,\ldots,X_n;\sigma^2)}{\partial (\sigma^2)^2}$ to be $-\frac{n}{2(\sigma^2)^2}+\frac{\sum_{i=1}^n(x_i - \mu)^2}{(\sigma^2)^3}$ which we can express as $$-\frac{n}{2\sigma^4}+\frac{1}{\sigma^4}\sum_{i=1}^n\Big(\frac{x_i - \mu}{\sigma}\Big)^2$$ Since $\sum_{i=1}^n\Big(\frac{X_i - \mu}{\sigma}\Big)^2$ given $\sigma^2$ possesses a $\chi^2$ distribution with $n$ degrees of freedom, we compute the Fisher information $\mathcal{I}(\sigma^2)$ to $$\mathcal{I}(\sigma^2)=-\frac{n}{2\sigma^4}+\frac{1}{\sigma^4}\mathbb{E}\Bigg(\sum_{i=1}^n\Big(\frac{X_i - \mu}{\sigma}\Big)^2\Bigg|\sigma^2\Bigg)=-\frac{n}{2\sigma^4}+\frac{n}{\sigma^4}=\frac{n}{2\sigma^4}$$ Evidently, the reciprocal of $\mathcal{I}(\sigma^2)$ is used as an estimator for the variance of the MLE of $\sigma^2,$ which in this case equals $\hat{\sigma}^2=\frac{1}{n}\sum_{i=1}^n(X_i-\mu)^2$. What's even more interesting is that $\frac{2\sigma^4}{n}$ is the exact variance of $\hat{\sigma}^2|\sigma^2$. To see this, first write $\hat{\sigma}^2=\frac{\sigma^2}{n}\sum_{i=1}^n\Big(\frac{X_i-\mu}{\sigma}\Big)^2$ and again noting that $$\sum_{i=1}^n\Big(\frac{X_i-\mu}{\sigma}\Big)^2\Big|\sigma^2 \sim \chi^2_{n}$$ it follows $$\mathbb{V}(\hat{\sigma}^2|\sigma^2)=\Big(\frac{\sigma^2}{n}\Big)^2\cdot \mathbb{V}\Bigg(\sum_{i=1}^n\Big(\frac{X_i-\mu}{\sigma}\Big)^2\Bigg|\sigma^2\Bigg)=\Big(\frac{\sigma^2}{n}\Big)^2\cdot 2n=\frac{2 \sigma^4}{n}$$ If we assume $H_0$ is true i.e. that $\sigma= \sigma_0$ then by CLT our MLE $\hat{\sigma}^2$ is asymptotically $N(\sigma_0^2,\frac{2\sigma_0^4}{n})$. This means $$\sqrt{n}\big(\hat{\sigma}^2-\sigma_0^2\big)\approx N\Big(0,(\sigma_0^2 \sqrt{2})^2\Big)$$ Applying the $\delta-$ method with $g(x)=\sqrt{x}$ yields $$\sqrt{n}\big(\hat{\sigma}-\sigma_0\big)=\sqrt{n}\big(g(\hat{\sigma}^2)-g(\sigma_0^2)\big)\approx N\Big(0,(\sigma_0^2 g'(\sigma_0^2) \sqrt{2})^2\Big)$$ The above is equivalent to $\sqrt{n}\Big(\hat{\sigma}-\sigma_0\Big)\approx N(0,\sigma_0^2/2)$ and an appropaite Wald statistic would be $$\frac{\hat{\sigma}-\sigma_0}{\sqrt{\sigma_0^2/2n}}$$