Solved – Test logistic regression model using residual deviance and degrees of freedom

chi-squared-testgeneralized linear modelhypothesis testinglogisticregression

I was reading this page on Princeton.edu. They are performing a logistic regression (with R). At some point they calculate the probability of getting a residual deviance higher than the one they got on a $\chi^2$ distribution with degrees of freedom equal to the degrees of freedom of the model. Copying-pasting from their website…

> glm( cbind(using,notUsing) ~ age + hiEduc + noMore, family=binomial)

Call:  glm(formula = cbind(using, notUsing) ~ age + hiEduc + noMore,      
     family = binomial) 

Coefficients:
(Intercept)     age25-29     age30-39     age40-49       hiEduc       noMore  
    -1.9662       0.3894       0.9086       1.1892       0.3250       0.8330  

Degrees of Freedom: 15 Total (i.e. Null);  10 Residual
Null Deviance:      165.8 
Residual Deviance: 29.92        AIC: 113.4

The residual deviance of 29.92 on 10 d.f. is highly significant:

> 1-pchisq(29.92,10)
[1] 0.0008828339

so we need a better model

Why does it make sense to compute 1-pchisq(29.92,10) and why does a low probability indicate that something is going wrong with their model?

Best Answer

They are using a deviance test shown below: $$ D(y) = -2\ell(\hat\beta;y) + 2\ell(\hat\theta^{(s)};y) $$

Here the $\hat β$ represents the fitted model of interest and $\hatθ(s)$ represents the saturated model. The log-likelihood for the saturated model is (more often than not) $0$, hence you are left with the residual deviance of the model they fitted ($29.92$). This deviance test is approximately chi-squared with degrees of freedom $n-p$ ($n$ being the observations and $p$ being the number of variables fitted). You have $n=16$ and $p=6$ so the test will be approximately $\chi^2_{10}$. The null of the test is that your fitted model fits the data well and there is no misfit—you haven't missed any sources of variation. In the above test you reject the null and, as a result, you have missed something in the model you fitted. The reason for using this test is that the saturated model will fit the data perfectly so if you were in the case where you are not rejecting the null between your fitted model and the saturated model, it indicates you haven't missed big sources of data variation in your model.

Related Question