Solved – Effect size for ANOVA oneway.test

anovaeffect-size

I am doing a one way analysis of variance to test whether the mean of an observation varies across groups. I'm using R's oneway.test rather than anova because I have thousands of groups and oneway.test is faster. How can I measure the effect size? Is there a way to calculate the $\eta^2$ using the output of oneway.test?

This is how I am doing my test:

oneway.test(o ~ g,
   data=data.frame(o=rep(c(1:100),10)+rnorm(1000,sd=0.1),g=rep(c(1:100),10)),
   var.equal = T)

I have 8500 groups, and each group has a different number of members, say between 3 and 10. Obviously with these many groups, I expect to get a very significant p-value. That's why I want to find the effect size, because for a large N, even a very small non-zero effect will give a significant pvalue.

Best Answer

In the absence of library function, one can manually calculate the $\eta^2. $Suppose the data is as follows:

set.seed(1); df <- data.frame(o=rep(c(1:10),4)+rnorm(40),g=as.factor(rep(5*c(1:10))))

I am computing the oneway test as

oneway.test(o ~ g, df, var.equal=T)

If I use aov, I can calculate $\eta^2$ as follows using the lsr package:

> lsr::etaSquared(aov(o~g,df))
     eta.sq eta.sq.part
g 0.9338765   0.9338765

Now, I will calculate the $\eta^2$ manually.

m.i = tapply(df$o, df$g, mean)
v.i = tapply(df$o, df$g, var)
n.i = tapply(df$o, df$g, length)

SS_within = sum((n.i-1) * v.i)
SS_between = sum(n.i * (m.i - mean(df$o))^2)
SS_total = (length(df$o)-1) * var(df$o)
SS_between/SS_total
[1] 0.9338765

which is the same as that calculated by the lsr package.

Related Question