ANOVA – Comparing Noncentrality Parameter in Repeated-Measures ANOVA: Formula vs. G*Power

anovagpowernon-centralrepeated measures

I am trying to figure out the correct expression for the noncentrality parameter $\lambda_{ws}$ for the within-subjects effect in a one-way Repeated-Measures ANOVA with $k$ trials/groups. Comparing the calculation of $\lambda_{ws}$ from G*Power with my own attempts (based on the literature) I noticed an inconsistency. I'm not sure where things are going wrong, so I was hoping someone here could shed some light.

I started from a paper by Potvin and Schutz (2000, p. 348), who provided the following formula:

$$\lambda_{ws} = n\frac{\sum_{j = 1}^k (\mu_j-\mu)^2}{\sigma_{wg}^2(1-\bar{\rho})},$$

where $\mu_j$ is the mean for trial/group $j$, $\bar{\rho}$ is the average correlation between scores in the trials/groups, and $\sigma_{wg}^2$ is the within-trial (within-group) variance (assumed to be constant across trials/groups). Thus, $\lambda_{ws}$ is comparable to $\lambda$ from a 'regular' (between-subjects) one-way ANOVA, but it is just multiplied by a factor $c = \frac{1}{1-\bar{\rho}}$:

$$\lambda = nf^² = n\frac{\eta^2}{1-\eta^2} = n\frac{\sum_{j = 1}^k (\mu_j-\mu)^2}{\sigma_{wg}^2}= \frac{1}{c}\lambda_{ws} = (1-\bar{\rho})\lambda_{ws} \Rightarrow \lambda_{ws}=\frac{\lambda}{(1-\bar{\rho})}.$$

However, when I tried to calculate $\lambda_{ws}$ using G*Power, it seemed to be using a different formula. I couldn't find that formula anywhere in the official manual, but I came across this tutorial document. On page 34, it notes that:

$$\lambda_{gpower} = \frac{knf^2}{(1-\bar{\rho})} = k\lambda_{ws},$$

so the two are off by a factor $k$. My question therefore is: which one is correct? My hunch is that the G*Power formula is off: it appears to 'double-count' the 'treatment' sums of squares in the numerator (first it sums over the groups, and then it multiplies the result by $k$ again). This would have made sense to me if there were a within-between-interaction, and if $k$ represented the number of levels of the between-subjects factor, but that's not the case here.

I'm just not sure, though.

EDIT

Thanks to @dipetkov's answer below I was able to see for myself that the G*Power and Potvin & Schutz formulas are identical (I will maintain $k$ to denote the number of trials; @dipetkov used $q$):

$$\lambda_{gpower}= \frac{Nkf^2}{1-\bar{\rho}}= \frac{npkf^2}{(1-\bar{\rho})} = \frac{nkf^2}{(1-\bar{\rho})}$$
$$=\frac{nk}{(1-\bar{\rho})}\frac{\eta^2}{(1-\eta^2)}$$
$$= \frac{nk}{(1-\bar{\rho})}\frac{\sum_{j = 1}^k n_j(\mu_j-\mu)^2}{\sum_{j = 1}^k \sum_{i=1}^n (y_{ij}-\mu_j)^2}$$
$$= \frac{nk}{(1-\bar{\rho})}\frac{\sum_{j = 1}^k n_j(\mu_j-\mu)^2}{\sum_{j = 1}^k n_j\sigma_{wg(j)}^2}$$
$$= \frac{nk}{(1-\bar{\rho})}\frac{n\sum_{j = 1}^k (\mu_j-\mu)^2}{kn\sigma_{wg}^2}$$
$$= \frac{n}{(1-\bar{\rho})}\frac{\sum_{j = 1}^k (\mu_j-\mu)^2}{\sigma_{wg}^2}$$
$$= \lambda_{ws}$$

Best Answer

TL;DR The G*Power formula is correct and the (Potvin and Schutz, 2000) formula is also correct. The missing step is how to adapt a formula for two-way ANOVA to the one-way layout.

You seem to mix the notation used by two different sources and the power formulas for one- and two-way repeated measures ANOVAs.

In the (Potvin and Schutz, 2000) paper [1], $n$ is the group sample size (number of subjects in each group) in a balanced design. In the G*Power tutorial [2], $N$ is the total sample size (total number of subjects). The relationship between the group sample size and the total size is simple: $N = pn$ where $p$ in the number of levels of the between-subjects factor A.

Note: (Potvin and Schutz, 2000) denote the number of A levels by $p$; the G*Power tutorial denotes it by $a$ and you denote it by $k$. I'll use the notation in the paper.

On pages 32—36 the G*Power tutorial presents results for a two-way $A_p \times B_q$ repeated measures (RM) ANOVA. You seem to ignore this fact as you are interested in a one-way repeated measures design. To an extent, you sweep the difference between the between-subjects and within-subject factors by referring to "the mean for trial/group $j$", implying that groups and trials are interchangeable.

In your question you reference the formula for the within-subject effect B. Let's show that the G*Power formula is equivalent to the (Potvin and Schutz, 2000) formula.

$$ \begin{aligned} \eta_{\text{G*Power}} = \frac{Nqf^2}{1-\rho} = \frac{npq\sum_{j=1}^q(\mu_j-\mu)^2/(q\sigma^2)}{1-\rho} = \frac{np\sum_{j=1}^q(\mu_j-\mu)^2}{\sigma^2(1-\rho)} = \lambda_B \end{aligned} $$

For completeness, let's also look at the formula for the between-subjects effect A. Aside: In the tutorial there is a typo: the effect size $f^2$ should be in the numerator, not the denominator.

$$ \begin{aligned} \eta_{\text{G*Power}} = \frac{Nqf^2}{1-(1-q)\rho} = \frac{npq\sum_{i=1}^p(\mu_i-\mu)^2/(p\sigma^2)}{1-(1-q)\rho} = \frac{nq\sum_{i=1}^p(\mu_i-\mu)^2}{\sigma^2(1-(1-q)\rho)} = \lambda_A \end{aligned} $$

So can you use G*Power to calculate sample size for a one-way repeated measures ANOVA? Yes: It corresponds to a two-way layout with one level for the A factor, ie, $p=1$. The noncentrality parameter is $\lambda = nqf^2/(1-\rho)$ just as given in your question.

[1] P. Potvin and R. Schutz. Statistical power for the two-factor repeated measures ANOVA. Behavior Research Methods, 32:347–356, 2012.
[2] G*Power tutorial by StatPower. Available online.

Power of F-tests for Gaussian linear models

General $F$-test

Any Gaussian linear model can be written $\boxed{Y=\mu+\sigma G}$ where $G$ has the standard normal distribution on $\mathbb{R}^n$ and $\mu$ is assumed to belong to a linear subspace $W$ of $\mathbb{R}^n$. Usually, the theory of Gaussian linear models treat them with the $Y=X\beta+\sigma G$ writing, corresponding to $W=\text{Im}(X)$. There are good reasons for that but the underlying geometry is clearer with the $\boxed{Y=\mu+\sigma G}$ treatment.

The so-called "ANOVA test" is a particular test of a F-test for a null hypothesis $H_0\colon\{\mu \in U\}$ where $U\subset W$ is a linear subspace. Actually the F-test exactly coincides with the likelihood-ratio test in this situation, and it is based on the Fisher statistic $$ F = \frac{{\Vert P_Z Y\Vert}^2/(m-\ell)}{{\Vert P_W^\perp Y\Vert}^2/(n-m)}, $$ where $Z=U^\perp \cap W$ is the orthogonal complement of $U$ in $W$, and denoting $m=\dim(W)$ and $\ell=\dim(U)$.

Obviously $\boxed{P_Z Y = P_Z \mu + \sigma P_Z G}$ and $\boxed{P_W^\perp Y = \sigma P_W^\perp G}$.

When $H_0\colon\{\mu \in U\}$ is true then $P_Z \mu = 0$ and therefore $$ F = \frac{{\Vert P_Z G\Vert}^2/(m-\ell)}{{\Vert P_W^\perp G\Vert}^2/(n-m)} \sim F_{m-\ell,n-m} $$ has the Fisher $F_{m-\ell,n-m}$ distribution. Consequently, from the classical relation between the Fisher distribution and the Beta distribution, $R^2 \sim {\cal B}(m-\ell, n-m)$.

In the general situation we have to deal with $P_Z Y = P_Z \mu + \sigma P_Z G$ when $P_Z\mu \neq 0$. In this general case one has ${\Vert P_Z Y\Vert}^2 \sim \sigma^2\chi^2_{m-\ell}(\lambda)$, where $\chi^2_{m-\ell}(\lambda)$ is the noncentral $\chi^2$ distribution with $m-\ell$ degrees of freedom and noncentrality parameter $\boxed{\lambda=\frac{{\Vert P_Z \mu\Vert}^2}{\sigma^2}}$, and then $\boxed{F \sim F_{m-\ell,n-m}(\lambda)}$ noncentral Fisher distribution. To compute $P_Z\mu$, note that $P_Z = P_W - P_U$ and $P_W\mu=\mu$, hence the only thing to compute is $P_U \mu$.

Power of the $F$-test

The power of the $F$-test depends on the significance level $\alpha$ which is chosen by the user. The critical value $c$ of the test is the value for which $\Pr_0(F>c)=1-\alpha$ where $\Pr_0$ denotes the probability under the null $H_0$. The power is then $\Pr(F>c)=1-\alpha$ where $\Pr$ denotes the probability under the unknown parameters. As we have seen, $\Pr$ only depends on $\lambda$, $n$, $m$ and $l$.

R code

The effect size in this situation is $\sqrt{\frac{\lambda}{n}}$, and it is more usual to take the effect size rather than the non-centrality parameter $\lambda$ as an input of the power. Thus I have written the following R function which returns the power using the effect size as an argument rather than the noncentrality parameter.

# alpha : significance level eff : effect size ; n : sample size ; m :
# number of parameters of the model ; l : number of parameters of the
# submodel H0 ;
Power <- function(alpha, eff, n, m, l) {
    df1 <- m - l
    df2 <- n - m
    c <- qf(1 - alpha, df1, df2)
    lambda <- eff^2 * n
    pow <- pf(c, df1, df2, ncp = lambda, lower.tail = FALSE)
    return(pow)
}

Example

Power(alpha = 5/100, eff = 0.5, n = 48, m = 5, l = 4)
## [1] 0.92303

The case of ANOVA

The pwr.anova.test() function of the pwr package calculates power of ANOVA for the balanced case only. For example, when there are $3$ groups, $4$ observations per group, the power for $\alpha=5\%$ and for an effect size of $0.5$ is

library(pwr)
pwr.anova.test(3, 4, 0.5, 5/100)
## 
##      Balanced one-way analysis of variance power calculation 
## 
##               k = 3
##               n = 4
##               f = 0.5
##       sig.level = 0.05
##           power = 0.24088
## 
## NOTE: n is number in each group

With the general notations of the $F$-test, here $U$ is a one-dimensional subspace corresponding to the assumptions that all group means are equal. The space $W$ has the same dimension as there are groups, here $3$. Thus using my function one get the power as follows:

Power(alpha = 5/100, eff = 0.5, n = 12, m = 3, l = 1)
[1] 0.24088

Solved – R: Power analysis for a 2 by 2 within-within ANOVA interaction effect [pwr::pwr.f2.test()]

I think that all of the ANOVA functions in pwr are 1-way. If you have the right data, you should be able to do your calculation using Superpower. It's available as an R package and a shiny app (which is a bit like G*Power).