Is mixed ANOVA the same thing as multilevel modeling? If not, how do they differ? I am trying to compare inter- and intra-individual differences and not sure which one is the better approach.
Solved – Mixed ANOVA vs. mixed models (multilevel modeling)
anovamixed modelmultilevel-analysis
Related Solutions
This post bases on personal experiences which might be specific to my data, so I'm not sure it qualifies as an answer.
I suggest to use simulations if possible to assess which method works best for your data. I did this and was surprised to find that tests (regarding parameters in the first level) based on multilevel modelling were outperforming any other method (power-wise), while retaining size even in small samples with few and unevenly sized "clusters". I am yet to find a paper that makes that point, and from how I see this is not really a niche topic and deserves more attention. I think it is fairly under-researched how different methods compare vis-a-vis finite-sample or few/uneven clusters.
Section 2.2.2.1 from lme4 book
Because each level of sample occurs with one and only one level of batch we say that sample is nested within batch. Some presentations of mixed-effects models, especially those related to multilevel modeling˜[Rasbash et˜al., 2000] or hierarchical linear models˜[Raudenbush and Bryk, 2002], leave the impression that one can only define random effects with respect to factors that are nested. This is the origin of the terms “multilevel”, referring to multiple, nested levels of variability, and “hierarchical”, also invoking the concept of a hierarchy of levels. To be fair, both those references do describe the use of models with random effects associated with non-nested factors, but such models tend to be treated as a special case.
The blurring of mixed-effects models with the concept of multiple, hierarchical levels of variation results in an unwarranted emphasis on “levels” when defining a model and leads to considerable confusion. It is perfectly legitimate to define models having random effects associated with non-nested factors. The reasons for the emphasis on defining random effects with respect to nested factors only are that such cases do occur frequently in practice and that some of the computational methods for estimating the parameters in the models can only be easily applied to nested factors
Best Answer
I don't have enough points to only respond in a comment, so I'll post as an answer.
I think that mixed ANOVA is something of a special case of multilevel modeling. Both can tell you about intra- and interindividual differences. One clear difference is that multilevel modeling uses maximum likelihood estimation, which gives you an advantage if you have missing data in your repeated-measure variable: mixed ANOVA will remove any incomplete cases from the analysis, whereas multilevel modeling makes use of all available information without needing to resort to listwise deletion (see Enders, 2011). Multilevel modeling won't give you an advantage if you have missing data on your predictors, however, in which case listwise deletion is also performed.
On the other hand, mixed ANOVA might be more accurate for small sample sizes. Maas & Hox (2005) showed that accuracy of parameter estimates (particularly of random effects, not so much of fixed effects) in multilevel modeling depends on sample size at level 2, but not so much at level 1; in your case, this would mean that you should probably hope for 50+ individuals in your sample. (FYI, if you're opting for multilevel modeling, more often than not you should use restricted maximum likelihood [REML] instead of full information maximum likelihood [FIML], particularly if you have small sample sizes. For large sample sizes, the two are equivalent, but for small sample sizes, restricted maximum likelihood is less biased. Be careful--REML is the default in lme4 and nlme in R, but FIML is the default in software like SAS, SPSS, and Mplus.)
If you have a number of missing data in your repeated-measure variable and have an adequate sample size, I would go with multilevel modeling.
References:
Enders, C. K. (2011). Missing not at random models for latent growth curve analyses. Psychological Methods, 16, 1-16.
Maas, C. J. M., & Hox, J. J. (2005). Sufficient sample sizes for multilevel modeling. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 1, 86-92.