Solved – Diagnostic plots for count regression

Question

What diagnostic plots (and perhaps formal tests) do you find most informative for regressions where the outcome is a count variable?

I'm especially interested in Poisson and negative binomial models, as well as zero-inflated and hurdle counterparts of each. Most of the sources I've found simply plot the residuals vs. fitted values without discussion of what these plots "should" look like.

Wisdom and references greatly appreciated. The back story on why I'm asking this, if it's relevant, is my other question.

Related discussions:

• Interpreting residual diagnostic plots for glm models?
• Assumptions of generalized linear models
• GLMs – Diagnostics and Which Family

Answer 1

Here is what I usually like doing (for illustration I use the overdispersed and not very easily modelled quine data of pupil's days absent from school from MASS):

1. Test and graph the original count data by plotting observed frequencies and fitted frequencies (see chapter 2 in Friendly) which is supported by the vcd package in R in large parts. For example, with goodfit and a rootogram:

   library(MASS)
   library(vcd)
   data(quine) 
   fit <- goodfit(quine$Days) 
   summary(fit) 
   rootogram(fit)
   

   or with Ord plots which help in identifying which count data model is underlying (e.g., here the slope is positive and the intercept is positive which speaks for a negative binomial distribution):

   Ord_plot(quine$Days)
   

   or with the "XXXXXXness" plots where XXXXX is the distribution of choice, say Poissoness plot (which speaks against Poisson, try also type="nbinom"):

   distplot(quine$Days, type="poisson")
   

2. Inspect usual goodness-of-fit measures (such as likelihood ratio statistics vs. a null model or similar):

   mod1 <- glm(Days~Age+Sex, data=quine, family="poisson")
   summary(mod1)
   anova(mod1, test="Chisq")
   

3. Check for over / underdispersion by looking at residual deviance/df or at a formal test statistic (e.g., see this answer). Here we have clearly overdispersion:

   library(AER)
   deviance(mod1)/mod1$df.residual
   dispersiontest(mod1)
   

4. Check for influential and leverage points, e.g., with the influencePlot in the car package. Of course here many points are highly influential because Poisson is a bad model:

   library(car)
   influencePlot(mod1)
   

5. Check for zero inflation by fitting a count data model and its zeroinflated / hurdle counterpart and compare them (usually with AIC). Here a zero inflated model would fit better than the simple Poisson (again probably due to overdispersion):

   library(pscl)
   mod2 <- zeroinfl(Days~Age+Sex, data=quine, dist="poisson")
   AIC(mod1, mod2)
   

6. Plot the residuals (raw, deviance or scaled) on the y-axis vs. the (log) predicted values (or the linear predictor) on the x-axis. Here we see some very large residuals and a substantial deviance of the deviance residuals from the normal (speaking against the Poisson; Edit: @FlorianHartig's answer suggests that normality of these residuals is not to be expected so this is not a conclusive clue):

   res <- residuals(mod1, type="deviance")
   plot(log(predict(mod1)), res)
   abline(h=0, lty=2)
   qqnorm(res)
   qqline(res)
   

7. If interested, plot a half normal probability plot of residuals by plotting ordered absolute residuals vs. expected normal values Atkinson (1981). A special feature would be to simulate a reference ‘line’ and envelope with simulated / bootstrapped confidence intervals (not shown though):

   library(faraway)
   halfnorm(residuals(mod1))
   

8. Diagnostic plots for log linear models for count data (see chapters 7.2 and 7.7 in Friendly's book). Plot predicted vs. observed values perhaps with some interval estimate (I did just for the age groups--here we see again that we are pretty far off with our estimates due to the overdispersion apart, perhaps, in group F3. The pink points are the point prediction $\pm$ one standard error):

   plot(Days~Age, data=quine) 
   prs  <- predict(mod1, type="response", se.fit=TRUE)
   pris <- data.frame("pest"=prs[[1]], "lwr"=prs[[1]]-prs[[2]], "upr"=prs[[1]]+prs[[2]])
   points(pris$pest ~ quine$Age, col="red")
   points(pris$lwr  ~ quine$Age, col="pink", pch=19)
   points(pris$upr  ~ quine$Age, col="pink", pch=19)
   

This should give you much of the useful information about your analysis and most steps work for all standard count data distributions (e.g., Poisson, Negative Binomial, COM Poisson, Power Laws).