Computing the LRT

hypothesis testinglog likelihoodstatistics

Suppose we have the data $(Y_i, x_i),\dots,(Y_n,x_n)$ such that $Y_i \sim N(\theta x_i,1)$. We can compute the MLE, which will yield
\begin{align*}
\hat{\theta}_{ML} = \dfrac{\sum_ix_iY_i}{\sum_ix_i^2}
\end{align*}

Now, we want to test the hypothesis $H_0 : \theta = \theta_0$ vs $H_1 : \theta \neq \theta_0$. First, we need to compute the LRT. However, I'm having trouble working out the problem.
\begin{align*}
\lambda(Y) &= \dfrac{\exp\bigg(-\frac{1}{2}\sum_{i=1}^n (y_i – \theta_0 x_i)^2\bigg)}{\exp\bigg(-\frac{1}{2}\sum_{i=1}^n (y_i – \hat\theta_{ML} x_i)^2\bigg)} \quad \text{(is this correct?)}\\
&= \exp\bigg(\frac{1}{2}\sum_{i=1}^n(Y_i^2 – 2\hat\theta_{ML}x_iY_i + \hat\theta_{ML}^2x_i^2) – (Y_i^2 – 2\theta_0x_iY_i + \theta^2x_i^2)\bigg) \\
&= \exp\bigg(\frac{1}{2}\sum2\theta_0x_iY_i – 2\hat\theta_{ML}x_iY_i +(\hat\theta_{ML}^2 – \theta_0^2)x_i^2\bigg)
\end{align*}

But this doesn't given nice results. What am I doing incorrect or what am I not seeing?

EDIT: forgot to mention the variance in the first sentence

Best Answer

\begin{align} & \sum_i \left( 2x_i y_i \left(\theta_0 - \widehat{\theta\,}\right) + \left(\widehat{\theta\,}^2 - \theta_0^2\right)x_i^2 \right) \\[10pt] = {} & \underbrace{\left( -2\theta_0 \sum_i x_i y_i + \theta_0^2 \sum_i x_i^2\right)}_\text{first term} {} - {}\underbrace{ \left(- 2\widehat{\theta\,} \sum_i x_i y_i + \widehat{\theta\,}^2 \sum_i x_i^2\right)}_\text{second term} \end{align} The first and second terms above have the form $$ -2\theta\sum_i x_i y_i + \theta^2 \sum_i x_i^2. $$ That function is a positive number multiplied by $$ -2\theta \frac{\sum_i x_i y_i}{\sum_i x_i^2} + \theta^2, \tag 1 $$ and that positive number is not a function of $\theta.$ And $(1)$ is equal to $$ \left( \frac{\sum_i x_i y_i}{\sum_i x_i^2} \right)^2 - 2\theta \frac{\sum_i x_i y_i}{\sum_i x_i^2} + \theta^2 + \big(\text{something} \big) \tag 2 $$ where the term labeled "something" simply cancels out the term added at the beginning, and the important thing to notice about it is that it also does not depend on $\theta.$ Line $(2)$ is equal to $$ \left( \frac{\sum_i x_i y_i}{\sum_i x_i^2} - \theta \right)^2 + \big( \text{something} \big) $$ where "something" is the same thing it was in line $(2).$

So the "first term" and the "second term" above add up to $$ \left( \frac{\sum_i x_i y_i}{\sum_i x_i^2} - \theta_0 \right)^2 - \left( \frac{\sum_i x_i y_i}{\sum_i x_i^2} - \widehat{\theta\,} \right)^2 \tag 3 $$ (and the thing labeled "something" has canceled out, although what really matters about it is that it does not depend on $\theta$).

Now observe that the second term in line $(3)$ is zero, and the first is $\left( \widehat{\theta\,} - \theta_0\right)^2.$

Related Question