Solved – GLMM – between, within and nested

Question

I'm not entirely sure of fitting the model for experiment we've made. The variables and relevant description are as follows:

• ID – participant ID
• Trial – 60 for each participant
• Memory – between subject binary factor
• State – within subject binary factor

---

• Correct – whether classification a participant made was correct or not
• Rating – the judgement made after each trial on four point Likert scale

Procedure brief: each participant (N=60) was randomly assigned to experimental or control group (Memory) and had 120 Trials (60 for State = 0 and 60 for State = 1). Each trial composed of perceptual classification (Correct) and judgment of how easy it was (Rating). The classification problem was randomly selected from two groups each trial (State).

I would like to calculate what impacts the performance (Correct) most – is it memory, state, a specific rating on a scale or any combination of above? I'm not interested in between subject variance, on the oposite, it is a random factor here. Also, it appears that there is bias in responses on Likert scales, so that part of variance should be excluded too.

The way I was thinking to approach this is generalized mixed linear model, but I'm not sure I'm doing it right; there is what I've got so far:

model = glmer(Correct ~ (1|ID/Rating) + Memory * State * Rating, data, family=binomial, 
              control = glmerControl(optimizer="bobyqa", optCtrl = list(maxfun=100000)))

Is this approach correct? I'll appreciate your input.

Relevant resources I used:

• Formulae in R: ANOVA and other models, mixed and fixed
• The Difference Between Crossed and Nested Factors
• When is it ok to remove the intercept in a linear regression model?
• Nested random factor with confounding (random?) variable

Answer 1

I don't see why Rating is nested within ID. This means that every every unique Rating belongs to one and only one ID which does not appear to be the case. If Rating is to be treated as random, then these should be crossed random effects. See the answer here for the difference between crossed and nested random effects and how to specify them.

That said, however, Rating should not be specified as a random intercept here because 1) there are only 4 levels, which is insufficient and 2) it is a likert scale item so the assumption of normality of random effects is hardly likely to hold.

So a better model would be

model = glmer(Correct ~ (1|ID) + Memory * State * Rating, data, family=binomial)

Lastly, note that there a several ways that you can treat Rating as an independent variable, see here, here and here for more details.