I am currently working on an exercise for statistics, but I am stuck at the last part of a question, (rejection regions).
We have $(Y_1, x_1), \dots ,(Y_n, x_n)$, where $\{Y_i\}_i$ are i.i.d. such that $$Y_i \sim \mathcal{N}(\theta x_i, 1), i \in \mathbb{N}.$$
I have shown that the maximum likelihood estimator is $\hat{\theta}_{ML} = \sum_{i = 1}^{n}x_i Y_i / \sum_{i = 1}^{n} x_i^2 $.
Furthermore, I found a $95\%$ confidence interval based on inverting $\hat{\theta}_{ML}$, namely $\left[\hat{\theta}_{ML} – z_{1-\alpha/2}\frac{1}{\sqrt{\sum x_i^2}}, \hat{\theta}_{ML} + z_{1-\alpha/2}\frac{1}{\sqrt{\sum x_i^2}}\right]$. However, I am not sure if this is necessary.
Now we are interested in testing $H_0: \theta = \theta_0$ against $H_1: \theta \neq \theta_0$.
I want to determine the rejection region $[0, c]$ associated with the log likelihood ratio as test statistic. I derived this test statistic, and found that $-2 \log(\lambda(Y)) \sim \chi^2_1$ under $H_0$. i.e., $$(\hat{\theta}_{ML} – \theta_0)^2\sum_{i = 1}^{n} x_i^2 \sim \chi^2_1 $$ under $H_0$.
So I don't really understand the $\chi^2_1$ distribution as yet, and this is where I am stuck. I think that $P($reject $H_0 | \theta = \theta_0) = \alpha$, however, I am quite stuck on how to apply this. Is there someone who could help me a little with this?
Thanks for your help in advance!
Best Answer
Assuming you observe $\mathbf Y=(Y_1,Y_2,\ldots,Y_n)$ where $Y_i\sim N(\theta x_i,1)$ independently for all $i$ and $x_i$ is fixed.
Indeed, MLE of $\theta$ is given by $$\hat\theta(\mathbf Y)=\frac{\sum_{i=1}^n x_i Y_i}{\sum_{i=1}^n x_i^2}$$
By the reproductive property of normal distribution, we have an exact distribution for the MLE:$$\hat\theta\sim N\left(\theta,\frac{1}{\sum_{i=1}^n x_i^2}\right)$$
In other words, $$\sqrt{\sum_{i=1}^n x_i^2}\left(\hat\theta-\theta\right)\sim N(0,1)$$
Using this pivot, a $100(1-\alpha)\%$ confidence interval for $\theta$ is $$I=\left[\hat\theta-\frac{z_{\alpha/2}}{\sqrt{\sum_{i=1}^n x_i^2}},\hat\theta+\frac{z_{\alpha/2}}{\sqrt{\sum_{i=1}^n x_i^2}}\right]$$
That is, $$P_{\theta}[\theta\in I]=1-\alpha\quad,\forall\,\theta$$
Or, $$P_{\theta}[\theta\in I^c]=\alpha\quad,\forall\,\theta$$
So for some $\theta_0$, $$P_{\theta_0}[\theta\in I^c]=\alpha$$
This gives the following critical region of a size $\alpha$ test for testing $H_0:\theta=\theta_0$ against $H_1:\theta\ne\theta_0$:
$$\left\{\mathbf Y:\hat\theta(\mathbf Y)<\theta_0-\frac{z_{\alpha/2}}{\sqrt{\sum_{i=1}^n x_i^2}}\quad\text{ or }\quad \hat\theta(\mathbf Y)>\theta_0+\frac{z_{\alpha/2}}{\sqrt{\sum_{i=1}^n x_i^2}}\right\}$$
Other tests can be derived of course but this gives you a test directly using the confidence interval $I$.