I did several independend t-tests. Some of them are significant, some of them are not.
Next, I computed Cohens $d$ for effect size.
For significant results, Cohens $d$ feels intuitively reasonable (How "large" is the found effect between both samples?). But how do I properly interpret Cohens $d$ in non-significant results?
Best Answer
Cohen's d can help to explain non-significant results: if your study has a small sample size, the chances of finding a statistically significant difference between the groups is unlikely, unless the effect size is large.
It's probably a good idea to include a confidence interval for your Cohen's d since the effect size based on your sample is still an estimate of the 'true' effect size.
This can be done easily in R using the
tes()
function in thecompute.es
package:Here is a useful short article on effect size thresholds: thresholds for interpreting effect sizes2.
And here is one on combining effect sizes with significance test interpretations (see especially sections 4 and 5): It's the Effect Size, Stupid: What effect size is and why it is important.
Here is a related post on how to interpret the confidence interval of Cohen's d in case you choose to find that: Why is the p-value for Cohen's $d$ not equal to the p-value of a t-test?.