AIC Comparison – Comparing Non-Nested Models with AIC

Question

Say we have to GLMMs

mod1 <- glmer(y ~ x + A + (1|g), data = dat)
mod2 <- glmer(y ~ x + B + (1|g), data = dat)

These models are not nested in the usual sense of:

a <- glmer(y ~ x + A + (1|g),     data = dat)
b <- glmer(y ~ x + A + B + (1|g), data = dat)

so we can't do anova(mod1, mod2) as we would with anova(a ,b).

Can we use AIC to say which is the best model instead?

Answer 1

The AIC can be applied with non nested models. In fact, this is one of the most extended myths (misunderstandings?) about AIC. See:

• Akaike Information Criterion

• AIC MYTHS AND MISUNDERSTANDINGS

One thing you have to be careful about is to include all the normalising constants, since these are different for the different (non-nested) models:

See also:

• Non-nested model selection

• AIC for non-nested models: normalizing constant

In the context of GLMM a more delicate question is how reliable is the AIC for comparing this sort of models (see also @BenBolker's). Other versions of the AIC are discussed and compared in the following paper:

• On the behaviour of marginal and conditional AIC in linear mixed models