Solved – When is a repeated measures ANOVA preferred over a mixed-effects model

anovamixed modelrepeated measures

In response to this question, regarding whether my design where I randomly presented participants with pictures from different categories was an example where I should use a repeated measures ANOVA, I got the answer that I should use a mixed-model instead, with one of the reasons being that I have two forms of dependencies: for subjects and for categories.

My question is now: Isn't it always the case that you have two dependencies in this way when doing this type of repeated measures design? That is, under what circumstances would a repeated-measures ANOVA be preferable to a mixed-effects modeling approach and why?

Best Answer

I'm not totally sure what actual model "repeated measures ANOVA" describes, but I think one general issue is whether to put random effects of any kind in a model rather than e.g. just adjust variance estimates to cover the induced dependencies (as in the Panel Corrected Standard Errors vs multilevel models debate in time series cross-sectional data analysis). So I'll have a go at that question first, then address yours.

Fixed and Random Effects

Two complementary principles about when to use a random rather than fixed effect are the following:

  1. Represent a thing (subject, stimulus type, etc.) with a random effect when you are interested using the model to generalise to other instances of that thing not included in the current analysis, e.g. other subject or other stimulus types. If not use a fixed effect.
  2. Represent a thing with a random effect when you think that for any instance of the thing, other instances in the data set are potentially informative about it. If you expect no such informativeness, then use a fixed effect.

Both motivate explicitly including subject random effects: you are usually interested in human populations in general and the elements of each subject's response set are correlated, predictable from each other and therefore informative about each other. It is less clear for things like stimuli. If there will only ever be three types of stimuli then 1. will motivate a fixed effect and 2. will make the decision depend on the nature of the stimuli.

Your questions

One reason to use a mixed model over a repeated effects ANOVA is that the former are considerably more general, e.g. they work equally easily with balanced and unbalanced designs and they are easily extended to multilevel models. In my (admittedly limited) reading on classical ANOVA and its extensions, mixed models seem to cover all the special cases that ANOVA extensions do. So I actually can't think of a statistical reason to prefer repeated measures ANOVA. Others may be able to help here. (A familiar sociological reason is that your field prefers to read about methods its older members learnt in graduate school, and a practical reason is that it might take a bit longer to learn how to use mixed models than a minor extension of ANOVA.)

Note

A caveat for using random effects, most relevant for non-experimental data, is that to maintain consistency you have to either assume that the random effects are uncorrelated with the model's fixed effects, or add fixed effect means as covariates for the random effect (discussed e.g. in Bafumi and Gelman's paper).

Related Question