When do I use Cohen's and when do I t-test? Probably in addition: What is the (conceptual) difference between them? Both tests are meant to study the difference between two distributions.
I just roughly know that Cohen's is used to calculate the effect size while t-test is meant to study whether there is a general difference between two distributions(?).
Formulas:
t-test: $ T = \frac{\bar{x} – \bar{y}}{ \sqrt{\frac{s_x^2}{n_x} + \frac{s_y^2}{n_y}} } $
with $ s $ as a variance.
Cohen's $d = \frac{\mu_1 – \mu_2}{ \sigma } $
I see there are differences but it also looks very similar. Isn't it?
Best Answer
Cohen's d seeks to tell you how big the standardized difference is between the two distributions. It's very popular in areas like psychology where I think there are no obvious units you can use to describe the difference. In medical stats, I could say (for example) that your HbA1c levels were on average 5mg different in the two groups, and wouldn't need to use Cohen's d.
The t-test is an attempt to tell you have enough evidence to reject the idea that the difference is non-zero. However, a non-zero difference could be, in practical terms, completely irrelevant. Also, don't forget you have to make technical assumptions when using the t-test, e.g. you default to assume the two groups have the same variance.
There are arguments that it is more useful to compare confidence or credible intervals estimated from the two samples.
There's an interesting article here: https://bmcresnotes.biomedcentral.com/articles/10.1186/s13104-015-1020-4