Solved – Why does the linear test statistic of GLM follow F-distribution

generalized linear model

As a MATLAB user, I have been using coefTest to perform linear hypothesis testing. For example in $y=\beta_0+\beta_1x_1+\beta_2x_2+\beta_3x_3$, if I want to test if $\beta_1=\beta_2$, then I can simply use a linear contrast $$C=\begin{bmatrix}0&1&-1&0\end{bmatrix}.$$
Then, the test statistic will follow an $F$-distribution, whereby I can compute my $p$-value.

  • Does this hold for all generalized linear models? In particular, I am concerned about the general linear model (Gaussian case) and the logistic regression (binomial case).

  • If so, why does the test statistic, despite so many different instantiations of GLM, always follow an $F$-distribution?

It seems that many sources just take this as granted, probably because this is too basic. Yet, I need to understand why so as to get assured enough to use it. I would sincerely appreciate if someone can point me to an authoritative book.

Best Answer

Why does the linear test static of GLM follow F-distribution?

It doesn't.

Then, the test statistic will follow an $F$-distribution [...] does this hold for all generalized linear models?

There's no result that establishes it in the general case, and indeed we can show (e.g. by simulation in particular instances) that it's not the case in general.

It holds for the Gaussian case, of course, but the derivation relies on the normality of the data. You can see it's not the case for logistic regression, since the data (and hence "F"-statistics based on the data) are discrete.

There is an asymptotic chi square result. This, combined with Slutsky's theorem should give us that the F-statistic will asymptotically be distributed as a scaled chi-square.

However, in sufficiently large samples (where how large "large" is will depend on a number of things), we might anticipate that The F distribution would still be approximately correct, since both the $F$ distribution being used to figure out p-values, and the actual distribution of the test statistic are both going to the same scaled chi-square distribution asymptotically.

We see the same issue with the common use of t-tests for parameter significance in GLMs (which many packages do) even though it's only t-distributed for the Gaussian case; for the others we only have an asymptotic normal result (but a similar argument for why the $t$ shouldn't do badly in sufficiently large samples can be made).

I don't have a good book suggestion. Some books give a handwavy argument for using the $F$ (some akin to mine above), others seem to ignore the need to justify it at all.

Related Question