Assuming I have two generalised linear models, one with only fixed effects and

another with fixed and random effects, how can I compare which model is most

parsimonious using AIC/BIC? I can only fit mixed-effect models with glmer, since

it returns an error when trying to fit a fixed effect model. I can only fit

fixed effect models with glm since this does not allow random effects. This

leaves me with two sets of models, one set fitted with glmer and another set

fitted with glm.

The accepted answer to the question Can AIC compare across different types of

model? states that (with my own emphasis):

It depends. AIC is a function of the log likelihood. If both types of model

compute the log likelihood the same way (i.e. include the same constant) then

yes you can, if the models are nested.

I'm reasonably certain that glm() and lmer() don't use comparable log

likelihoods.The point about nested models is also up for discussion. Some say AIC is only

valid for nested models as that is how the theory is presented/worked

through. Others use it for all sorts of comparisons.

The DRAFT r-sig-mixed-models FAQ states that:

Can I use AIC for mixed models? How do I count the number of degrees of

freedom for a random effect?Yes, with caution.

This page then follows by listing a number of caveats and suggestions such as

using modified AIC calculations or `REML=FALSE`

.

As a novice in this field, I have found that reading about both the use of

AIC/BIC for comparing models, and using mixed effect models for group effects

to be very eye-opening and also intuitively "common sense" approaches. How to

put these two approaches together has left me feeling like I've fallen through

the cracks somewhat, there doesn't seem to be a clear answer. Surely if I

follow a the AIC/BIC parsimony approach, then I should compare the fixed-effect

model with mixed effect model with AIC to see if the random effect deserves to

be there?

Am I missing something obvious? How are mixed-effect and fixed-effect

generalised linear models usually compared?

## Best Answer

As far as I can tell, you can compare the likelihoods of

`glmer()`

and`glm()`

models, at least for`family=binomial`

(haven't tested this for other families). If the variance components are estimated to be zero, then the likelihood should be identical and that is clearly the case. Here is an example to illustrate this:The last three lines yield:

So, the (log)-likelihoods are identical, since the

`id`

level variance component is estimated to be zero. The AIC value of the mixed-effects model is therefore 2 points larger, as expected (since the model has one more parameter).One thing to note though: The default for

`glmer()`

is`nAGQ=1`

, which means that the Laplace approximation is used. Let's use "proper" adaptive quadrature:This yields:

The variance component is still estimated to be zero and the (log)-likelihoods are identical. But

`anova()`

spits out an error that indicates that these models should not not be compared against each other.