Solved – What are Cohen’s rules of thumb for interpreting $η^2$

anovaeffect-size

I'm aware of the limitations of these simple rules of thumb, but in the research context in which I'm currently working, I can't avoid them.

This Cambridge University website cites the rules of thumb of Cohen (1988) for $η^2$ as being

0.01 = small effect
0.06 = medium effect
0.14 = large effect

Their figures seem to come from the table on p283, but it seems to me that straightfowardly reading the values off that table isn't right because the table represents $η^2$ as a function of f. However, according to my recording of the original document here (specifically p283-284) I worked it out that an $η^2$ of

0.100 = small effect
0.243 = medium effect
0.372 = large effect

The Cohen text is not very easy to read, and so I wonder that perhaps I have made a mistake. Am I missing something obvious here?

Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd edition). Hillsdale, New Jersey: L.

Best Answer

Both of them are right; it's easy to see the answer returns the square.

eta = 0.100  --->  (eta)^2 =0.01   small effect
eta = 0.243  --->  (eta)^2 =0.06   medium effect
eta = 0.372  --->  (eta)^2 =0.14   large effect

Effect sizes for group mean differences

In general, I find standardised group mean differences (e.g., Cohen's d) a more meaningful effect size measure within the context of group differences. Measures like eta square are influenced by whether group samples sizes are equal, whereas Cohen's d is not. I also think that the meaning of d-based measures are more intuitive when what you are trying to quantify is a difference between group means.
The above point is particularly strong for the case where you only have two groups (e.g., the effect of treatment versus control). If you have more than two groups, then the situation is a little more complicated. I can see the argument for variance explained measures in this case. Alternatively, Cohen's $f^2$ is another option.
A third option is that within the context of experimental effects, even when there are more than two groups, the concept of effect is best conceptualised as a binary comparison (i.e., the effect of one condition relative to another). In this case, you can once again return to d-based measures. The d-based measure is not an effect size measure for the factor, but rather of one group relative to a reference group. The key is to define a meaningful reference group.
Finally, it is important to remember the broader aim of including effect size measures. It is to give the reader a sense of the size of the effect of interest. Any standardised measure of effect should assist the reader in this task. If the dependent variable is on an inherently meaningful scale, then don't shy away from interpreting the size of effect in terms of that scale. E.g., scales like reaction time, salary, height, weight, etc. are inherently meaningful. If you find, as I do, eta squared to be a bit unintuitive within the context of experimental effects, then perhaps choose another index.

Eta squared versus partial eta squared

Partial eta squared is the default effect size measure reported in several ANOVA procedures in SPSS. I assume this is why I frequently get questions about it.
If you only have one predictor variable, then partial eta squared is equivalent to eta squared.
This article explains the difference between eta squared and partial eta squared (Levine and Hullett Eta Squared, Partial Eta Squared..).
In summary, if you have more than one predictor, partial eta squared is the variance explained by a given variable of the variance remaining after excluding variance explained by other predictors.

Rules of thumb for eta squared and partial eta squared

If you only have one predictor then, eta squared and partial eta squared are the same and thus the same rules of thumb would apply.
If you have more than one predictor, then I think that the general rules of thumb for eta squared would apply more to partial eta squared than to eta squared. This is because partial eta squared in factorial ANOVA arguably more closely approximates what eta squared would have been for the factor had it been a one-way ANOVA; and it is presumably a one-way ANOVA which gave rise to Cohen's rules of thumb. In general, including other factors in an experimental design should typically reduce eta squared, but not necessarily partial eta squared due to the fact that the second factor, if it has an effect, increases variability in the dependent variable.
Despite what I say about rules of thumb for eta squared and partial eta squared, I reiterate that I'm not a fan of variance explained measures of effect size within the context of interpreting the size and meaning of experimental effects. Equally, rules of thumb are just that, rough, context dependent, and not to be taken too seriously.

Reporting effect size in the context of significant and non-significant results

In some sense an aim of your research is to estimate various quantitative estimates of the effects of your variables of interest in the population.
Effect sizes are one quantification of a point estimate of this effect. The bigger your sample size is, the more close, in general, your sample point estimate will be to the true population effect.
In broad terms, significance testing aims to rule out chance as an explanation of your results. Thus, the p-value tells you the probability of observing an effect size as or more extreme assuming the null hypothesis was true.
Ultimately, you want to rule out no effect and want to say something about the size of the true population effect. Confidence intervals and credibility intervals around effect sizes are two approaches that get at this issue more directly. However, reporting p-values and point estimates of effect size is quite common and much better than reporting only p-values or only effect size measures.
With regards to your specific question, if you have non-significant results, it is your decision as to whether you report effect size measures. I think if you have a table with many results then having an effect size column that is used regardless of significance makes sense. Even in non-significant contexts effect sizes with confidence intervals can be informative in indicating whether the non-significant findings could be due to inadequate sample size.

Confidence Interval – Calculating Effect Size Confidence Interval for Mann-Whitney U-Test

One choice of effect size for the Mann-Whitney U test is the common language effect size. For the Mann-Whitney U, this is the proportion of sample pairs that supports a stated hypothesis.

A second choice is the rank correlation; because the rank correlation ranges from -1 to +1, it has properties that are similar to the Pearson r. In addition, by the simple difference formula, the rank correlation is the difference between the common language effect size and its complement, a fact that promotes interpretation. For example, if there are 100 sample pairs, and if 70 sample pairs support the hypothesis, then the common language effect size is 70%, and the rank correlation is r = .70 = .30 = .40. A clear discussion of the common language effect size and of four formulas to compute the rank correlation is given by Kerby in the journal Innovative Teaching: Kerby (2014) Innovative Teaching

By the way, though the paper does not mention it, I am fairly certain that Somers d and the rank correlation for the Mann-Whitney are equivalent.

Best Answer

Related Solutions

Solved – How to interpret and report eta squared / partial eta squared in statistically significant and non-significant analyses

Effect sizes for group mean differences

Eta squared versus partial eta squared

Rules of thumb for eta squared and partial eta squared

Reporting effect size in the context of significant and non-significant results

Confidence Interval – Calculating Effect Size Confidence Interval for Mann-Whitney U-Test

Related Question