Goodness-of-Fit – Deviance vs Pearson Goodness-of-Fit

chi-squared-testdeviancegeneralized linear modelgoodness of fitregression

I am trying to come up with a model by using negative binomial regression (negative binomial GLM). I have a relatively small sample size (greater than 300), and the data are not scaled. I noticed that there are two ways to measure goodness of fit – one is deviance and the other is the Pearson statistic.
How can I determine which goodness-of-fit measure to use? Are there some criteria that I can take a look at in selecting the goodness-of-fit measure?

Best Answer

The goodness-of-fit test based on deviance is a likelihood-ratio test between the fitted model & the saturated one (one in which each observation gets its own parameter). Pearson's test is a score test; the expected value of the score (the first derivative of the log-likelihood function) is zero if the fitted model is correct, & you're taking a greater difference from zero as stronger evidence of lack of fit. The theory is discussed in Smyth (2003), "Pearson's goodness of fit statistic as a score test statistic", Statistics and science: a Festschrift for Terry Speed.

In practice people usually rely on the asymptotic approximation of both to the chi-squared distribution - for a negative binomial model this means the expected counts shouldn't be too small. Smyth notes that the Pearson test is more robust against model mis-specification, as you're only considering the fitted model as a null without having to assume a particular form for a saturated model. I've never noticed much difference between them.

You may want to reflect that a significant lack of fit with either tells you what you probably already know: that your model isn't a perfect representation of reality. You're more likely to be told this the larger your sample size. Perhaps a more germane question is whether or not you can improve your model, & what diagnostic methods can help you.