Solved – Effect size calculations in mixed ANOVA design for power analysis

anovaeffect-sizestatistical-powervariance

I need to calculate effect size for a mixed ANOVA design. I need it because I'm using GPower3 to calculate a rough sample size for a full study. I have pilot data. Based on this page:

http://www.xlstat.com/en/features/statistical-power-analysis-of-variance-anova.htm

and the GPower3 software, it looks like there are 3 ways to calculate an effect size:

  • eta²
  • use variances
  • use group means

Is there any rational for picking one over another? Since I have the pilot data, I guess I was just going to use the variances to calculate the effect size. The formula I have for that is:

f (effect size) = sqrt(Variance Explained / Variance Error)

Based on the summation rule:

Total Variance = Variance Explained + Variance Error

Is this correct?

If I have unequal group sizes, could I use the pooled variance formula for the Variance Error term?

I also noticed that this effect size formula using the variances is quite similar to the F statistic:

F = Variance Explained / Variance Error

Is this correct and/or is there a relationship between them?

Thanks in advance.

Best Answer

I am not familiar with Gpower3 so this does not exactly address your question, but here is another approach that you may want to consider.

For power and sample size studies more complicated than simple 2 sample tests I prefer to use simulation rather than prepackaged routines. The basic steps are:

  1. Think through what your data will look like (structure, means, variances, distributions, ...)
  2. Think through how you will analyse the data
  3. Generate data based on current assumptions and analyze it
  4. Repeate step 3 many times and compute the percentage of times that the null is rejected (this is the power)
  5. Repeate steps 3 and 4 for different sets of assumptions

One of the advantages of this approach is that you know exactly what assumptions are being made and what the nature of the data and analysis is. The main disadvantage is that it takes a bit more work and time than just plugging in a few numbers to a canned routine.

Related Question