Cohen’s d – Is the Squared Cohen’s $f$ the Same as Cohen’s $f^2$

cohens-dinteractionregressionstatistical-powerterminology

Question

Is the squared version of Cohen's $f$ the same as Cohen's $f^2$? Or do they only share the same letter by coincidence?

Explanation

At first, I wondered if one can simply square Cohen's $f$ to obtain the Cohen's $f^2$. For example, papers often report Cohen's $d$, but power analyses (for regression) often require $f^2$ as estimate of the effect size. According to this source, it is possible to convert Cohen's $d$ to $f$ with the following formula :

$f = d/2$

Therefore, if one were to square the resulting $f$, would one be able to use this value as estimated effect size in a power analysis?

I would think that you cannot do that and that they are not related because they are part of different families and use different calculations ($f^2$ is part of the correlation family, while $f$ is part of the difference family). I was speaking with a statistician friend recently who was unsure himself because they're both related to regression under the hood. So I just wanted to confirm this with the community?

Best Answer

The effect size index $f^2$ is defined by Cohen as $\frac{R^2}{1-R^2}$ in a regression model where $R^2$ is the coefficient of determination (the % of variance explained by the model). Cohen emphasized the fact that is fundamentally identical to the $f$ index, hence you can take the square root of $f^2$ to get your estimate for $f$ (Statistical Power Analysis for the behavioral Sciences, 2nd ed., Lawrence Erlbaum Associates, 1988; §9.2, p. 410). It is just easier to work with a squared index when you already work with squared correlation values.

Sometimes, it is also computed as $\frac{\eta^2}{1-\eta^2}$ or $\frac{\eta_\text{partial}^2}{1-\eta_\text{partial}^2}$, especially when the authors focus on ANOVA. This is discussed later in $9.2.1, but you will find the same explanation on the UCLA website.

Related Solutions

Solved – Determination of effect size for a repeated measures ANOVA power analysis

Assuming you are going to average the first 12 months to form a baseline measure and the second 12 months to form as a follow-up measure, your problem reduces to a repeated measures t-test.

G*Power

You might want to check out the following menu in G*Power 3: Tests - Means - Two Dependent Groups (matched pairs). Use A priori, $\alpha=.05$, Power = 0.90. Use the Determine button to determine effect size. This requires that you can estimate time 1 and 2 means, sds, and correlation between time points.

If you know nothing about the domain, based on my experience in psychology, I'd start with something like

M1 = 0, SD1 = 1, SD2 = 1
correlation = .60

This means that M2 is basically a between subjects cohen's d.

You could then examine a few different values of M2 such as 0.2, 0.3, ... 0.5, ... 0.8, etc. Cohen's rules of thumb suggest 0.2 is small, 0.5 is medium, and 0.8 is large.

UCLA has a tutorial on doing a power analysis on a repeated measures t-test using R.

Side point

As a side point, you might want to consider having a control group.

Statistical Power – Differences and Relation Between Retrospective Power Analysis and A Posteriori Power Analysis

Assume for simplicity that your model is defined by only one parameter $\theta$. The power is the function $\theta \mapsto \Pr(\text{reject } H_0 \mid \theta)$, which depends on the sample size $n$.

In Retrospective Power Analysis, you only plug in your estimate $\theta$: you look at the value $\Pr(\text{reject } H_0 \mid \hat\theta)$ at the power function at $\theta=\hat\theta$, with the same sample size $n$. It answers the question: "what would be the probability that I would obtain significant results if $\theta$ were $\hat\theta$" ? As said in your text this question is rather useless because there is a one-to-one correspondance between the $p$-value and the restrospective power $\Pr(\text{reject } H_0 \mid \hat\theta)$.
For instance consider a binomial experiment with proportion parameter $\theta \in [0,1]$ and the hypothesis $H_0\colon\{\theta=0\}$. Obviously the power increases when $\theta$ increases. And obviously the $p$-value decreases when $\hat\theta$ increases. Consequently the lower $p$-value, the higher RP (retrospective power). A couple of years ago I wrote a R code for the case of Fisher tests in classical Gaussian linear models. It is here. There's a code using simulations for the one-way ANOVA example and a code for the general model provding an exact calculation of RP in function of the $p$-value and the design parameters. I called my function PAP() because "Puissance a posteriori" is the French translation of RP and PAP is also an acronym for "Power Approach Paradox". The cause of the decreasing correspondence between $p$ and RP for Gaussian linear models is intuitively the same as for the binomial experiment: if $\theta$ is "far from $H_0$" then the power at $\theta$ is high, and if $\hat\theta$ is "far from $H_0$" then the $p$-value is small. Theoretically this is a consequence of the fact that the noncentral Fisher distributions are stochastically increasing in the noncentrality parameter (see this discussion about noncentral $F$ distributions in Gaussian linear models). In fact here the noncentrality parameter plays the role of $\theta$ (is it the so-called effect size ? I don't know).
I claimed "RS is rather useless because of the correspondence with $p$" because this decreasing correspondence with $p$ means that having a high RP is equivalent to having a small $p$, and vice-versa. But the more serious problem is the misinterpretation of RP; for instance, I have found such claims in the literature:

$H_0$ is not rejected and RP is high, so the decision of the test is significant.
$H_0$ is not rejected, it is not surprising because RP is low.
$H_0$ is rejected (so the decision is significant) and RP is high, so the decision is even more significant.

Respectively replace "RP is high" and "RP is low" with "$p$ is low" and "$p$ is high" in the three claims above and you will see that they are either useless, wrong, or puzzling.
From a more "philosophical" perspective, RP is useless because why would we mind about the probability that rejection of $H_0$ occurs once the experiment is done ?
See also here a funny but clever retrospective power online calculator ;-)

The paragraph A Posteriori Power Analysis says nothing about the choice of $\theta$, but it emphasizes the main difference with the retrospective power: here the goal is to use the information issued from your first experiment to evaluate the power of a future experiment, focusing on the sample size. A sensible approach to evaluate this power is to consider your estimate $\hat\theta$ as a "guess" of the true $\theta$ and also to consider the uncertainty about this estimate. There is a natural way to do so in Bayesian statistics, namely the predictive power, which consists to average the possible values of $\Pr(\text{reject } H_0 \mid \theta)$ for various values of $\theta$, according to some distribution (the posterior distribution in Bayesian terms) representing the knowledge and the uncertainty about $\theta$ resulting from your first experiment. In the frequentist framework you could consider the values of the power evaluated at the bounds of your confidence interval about $\theta$.

Best Answer

Related Solutions

Solved – Determination of effect size for a repeated measures ANOVA power analysis

Statistical Power – Differences and Relation Between Retrospective Power Analysis and A Posteriori Power Analysis

Related Question