Binomial Count Data – Should glmer, lmer Be Used or Just Average It All?

binomial distributionlme4-nlme

I'm new to both mixed-effects models and R, so please excuse me if this is a bit of a silly question! I'm struggling with choosing the best method to analyse my data for a speech perception study, and would appreciate some help!

Some info about the data:

In the study, participants listen to sentences in different accents and embedded in different noise levels, and then repeat what they understand. So for each sentence there is the number of words correctly repeated, and the total number of words in each sentence.

I'm trying to figure out how to analyse this data in a mixed-effect model. I thought a binomial glmer would be best, as it's count data, but I had no idea if I was doing this right (particularly specifying the proportion of words correctly repeated), so I tried the same model (without interactions) in glmer and lmer.

glmer:

> mixed.glmer<-glmer(formula=cbind(Correct, Max-Correct)~Accent+Noise_Factor+Accent+(1|Participant), data=noise.data, family="binomial")

lmer:

> all.lmer<-lmer(Correct/Max~Accent+Noise_Factor+(1|Participant), data=noise.data, REML=FALSE)

Then I compared the models:

>               Df     AIC     BIC  logLik deviance Chisq Chi Df Pr(>Chisq)    
> all.glmer      6 13441.2 13480.1 -6714.6  13429.2                            
> all.lmer       7  2564.6  2610.0 -1275.3   2550.6 10879      1     <2e-16 ***

Which seems to suggest that the lmer model is a much better fit, but surely the data isn't being modelled well by linear methods?

Then, a colleague suggested that I average across all noise levels for each accent (so I can treat the data using linear methods), giving only three data points per participant. So I tried this model, which seems to be a much better fit than either of the above models with all of the data:

> average.m1<-lmer(Average.score~Accent+(1|Participant), data=mixed.model, REML=FALSE)
> AIC       BIC    logLik  deviance 
> -102.1759  -93.1426   56.0880 -112.1759

However, this seems to be throwing away a lot of data (particularly now that the noise factor has disappeared), even if the model does fit better – I thought the whole point of mixed models was to avoid doing the averaging step, like I'd need to for an ANOVA. This seems like very much like the easy way out of this problem!

So my questions are:

  1. Should I bother trying to fit a model to the whole dataset, or go with the easy way and average it?

  2. Am I right to think that for the whole data set, the lmer model is a better fit than the glmer model, or is this like comparing apples and oranges?

  3. If I do use the gmler model, am I specifying the DV correctly? It should be the proportion of correct responses (i.e.: Correct/Max). I'm finding it very, very hard to find instructions that I can actually understand!

Any help would be highly appreciated!

Best Answer

Let's start with the first, best, model:

mixed <- glmer(cbind(Correct, Max-Correct) ~ Accent + Noise_Factor + (1 | Participant),
               data=noise.data, family="binomial")

where I've tidied the syntax and removed the extra reference to Accent. This seems reasonable as a starting point.

Should you ignore (by leaving out) Noise_Factor?

Surely not, if you don't have to. You put it in there for a reason I presume. Consequently the right hand side of the third model seems underspecified. In the event that it truly has no effect itself and it also does not moderate the effects of anything else then leaving it out would be reasonable, but I assume you are not in a position to assert something that strong. Certainly you don't have to.

Is the data set fit better by one of the first or second model?

That's very hard to say. The DV in the first and second models are not the same and the models are not nested so any analysis of deviance (or even an R-squared) comparison doesn't seem to make any sense.

Are you specifying the DV in the first model correctly?

Yes.

Recommendation

In sum, I'd recommend you start with model one as above. If you wonder about interactions then you can use the anova function to compare models with and without them because they are nested and have the same DV.

You might look also to see whether the effect of other covariates vary by subject. Currently subjects vary in the ratio of correct to incorrect responses they make on average, but they might possibly differ in more than that. This is quite straightforward to check in lme4.

Related Question