Solved – How to deal with unbalanced group sizes in mixed design analysis

anovamixed modelrrepeated measures

I have 2 x 2 x 2 mixed design with two between subject factors (sex, organizer) and one within subjects factor (task). The group sizes of the 'sex'-factor is unequal. When I perform a factorial repeated measures ANOVA, I get the following warning message:

Warning: Data is unbalanced (unequal N per group). Make sure you
specified a well-considered value for the type argument to ezANOVA().

I used the following model in R:

model <- ezANOVA(data=df, dv=.(top_start), wid=.(id), between=.(sex, org),
                         within=.(task), type = 3, detailed = TRUE)

I used type = 3 for the Anova because, as I understood, it is suited for unbalanced group sizes.

I have the following questions:

Do I need to code contrasts when all my factors have only two levels?
Did I use the right type of Anova?
Are there other ways to do this analysis?

Best Answer

If you use type 3 for ANOVAs it is critical in R that you set the contrast to effect coding (i.e., "contr.sum").

The default contrast in R is dummy coding (or in R parlance, treatment coding) in which 0 represents the first factor level. This doesn't make too much sense when having interactions as explaind on the page I linked to.

To set effect coding, run the following:

options(contrasts=c("contr.sum","contr.poly"))

Alternatively, you can use the afex package, which has similar goals as ez, with the difference that it automatically sets the contrasts to effects coding and uses type 3 as default.

Related Solutions

R – How to Specify Mixed ANOVA with Multiple Repeated Measures and Covariate in R

First of all, you really need to make sure, that adding a covariate makes any sense. Adding a covariate only makes sense, if the covariate is not correlated with the independent variable (i.e., if there is no a priori reason to assume that being in either of the pretrain or trainign conditions affects study). To make sure to you really understand this I highly recommend reading the following brief paper:

Miller, G. A., & Chapman, J. P. (2001). Misunderstanding analysis of covariance. Journal of Abnormal Psychology, 110(1), 40–48. doi:10.1037/0021-843X.110.1.40

Then to run the ANOVA and compare it with an ANOCVA, you can use aov.car or ez.glm from afex (disclaimer: I am the author of afex), as in the following:

require(afex)
#ANOVA
ez.glm("subjid", "accuracy", D, within=c("section","probtype"), 
       between= c("pretrain","training"))

#ANCOVA
ez.glm("subjid", "accuracy", D, within=c("section","probtype"), 
       between= c("pretrain","training"), covariate= "study",
       factorize = FALSE)

A few more things to note:

Your covariate is not centered on 0, this makes afex throw a warning. As it is not in a between-subjects interaction, you should be able to safely ignore that.
You need to set factorize = FALSE when including a numerical covariate as the default behavior of ez.glm and aov.car is to automatically factorize all variables.
The results from ezANOVA with between_covariate should be very similar, however, the last time I checked, the calculation of the df was wrong (ezANOVA, does incorrectly not use one df per numerical covariate, rather covariates do not change the df).
Probably, ANOVA is NOT the correct tool for your analysis. You should definitely read the following excellent paper on why accuracy shouldn't be analyzed with ANOVA:
Jaeger, T. F. (2008). Categorical data analysis: Away from ANOVAs (transformation or not) and towards logit mixed models. Journal of Memory and Language, 59(4), 434–446. doi:10.1016/j.jml.2007.11.007

Solved – Power Analysis for a mixed two-way ANOVA with unqual sample sizes

The simulation method to calculate the sample size when the statistical methods is not simple.

Write the statistical model mathematically.
Generate the sample according to the model with sample size N.
Fit the model, perform the test, and record the rejection or acceptance of hull hypothesis.
Repeat step 2 and 3 n (generally I used 5000) times.
Calculate the power by (# of rejections)/n.
If power is too higher, decrease sample size N, repeat 2 - 5. If power is too lower, increase sample size N, repeat 2 - 5.

Given the current computer speed, after you finish the programming, it will not take days to find the suitable sample size.

If the method of statistical test is relatively new, it is better to generate the data under the null hypothesis, such that you can verify the alpha level of the test.