Solved – Likelihood ratio tests

likelihood-ratioregression

Is likelihood ratio test ($F$-test) of significance of difference of two linear models the same as chi-square test of difference of $-2\log L$?

SAS PROC GLM produces $F$-statistics and PROC MIXED $-2\log L$.

Best Answer

I think that you might be confusing an extra-sum-of-squares F-test with a likelihood ratio test. Although, both are used to compare two models.

A likelihood ratio statistic, denoted by $\Lambda$, is given by

$$\Lambda = \frac{L\text{(reduced model})}{L(\text{full model})}$$

Taking $-2\log\Lambda$ produces a statistic that has $\chi^2_{d.f(\text{reduced model})-d.f(\text{full model})}$ distribution. That is to say that taking $-2\log$ of the $\Lambda$ gives you a $\chi^2$ distribution.

I have not used SAS so I cannot comment on the output, but I hope that I have been able to answer your question.

Note: that $\Lambda$ is equivalent to your L

Janne: For linear regression you could use either the likelihood ratio test or the extra-sum-squares F-test and you should end up with the same p-value. Despite, this they are not the same thing.

As has been mentioned above the likelihood ratio test produces a statistic that has $\chi^2_{d.f(\text{reduced model})-d.f(\text{full model})}$ distribution. Where as an extra-sum-of-squares F-test, given by

$$F = \frac{(SSR_{\text{reduced model}}-SSR_{\text{full model}})/d.f_{\text{reduced model}} - d.f_{\text{full model}}}{\hat{\sigma}^2_\text{full model}}$$

producing a statistic that has $F_{d.f(\text{reduced model})-d.f(\text{full model}),d.f(\text{full model})}$ distribution. Where SSR is the sum of squared residuals and $\hat{\sigma}^2$ is our standard estimate.

Related Question