I would be wary of using the published value of r, 0.47, as the basis for your sample size calculations.
What if the true correlation is say 0.25? If that were the true population correlation, would you want your study to find a "significant" result? If so, compute the sample size for r = 0.25 (or even smaller). More generally, try to find the sample size that can detect (with reasonable power) the smallest effect (correlation coefficient for this example) you would care about.
I was not able to reproduce the results you got from WebPower using the pilot data you supplied. I was able to reproduce your R code however.
You are correct that you can't use the $\eta^2$ for Cohen's f, but $f^2 = \frac{\eta^2}{1-\eta^2}$
"However, how should I compute the effect size from the pilot study" - use the $\eta^2$ from the pilot study.
"Why are there interaction effect sizes, i.e, the effect size for group x vs group y?" Those are the effect sizes for the pair-wise comparisons (if you were using a t-test or a TukeyHSD)
require(dplyr)
require(reshape2)
pilot <- data.frame(option1 = c(6.3, 2.8, 7.8, 7.9, 4.9),
option2 = c(9.9, 4.1, 3.9, 6.3, 6.9),
option3 = c(5.1, 2.9, 3.6, 5.7, 4.5),
option4 = c(1.0, 2.8, 4.8, 3.9, 1.6))
pilot2 <- pilot %>%
reshape2::melt(value.name = "y") %>%
dplyr::rename("option" = "variable")
lm1 <- lm(y ~ option, data = pilot2)
aov1 <- aov(lm1)
means <- apply(pilot, 2, mean)
vs <- apply(pilot, 2, var)
# cohen's f for overall anova
# eta^2 = SSR / SST
eta.sq <- anova(lm1)$`Sum Sq`[2] / sum(anova(lm1)$`Sum Sq`)
f <- sqrt(eta.sq / (1-eta.sq))
# cohen's d for pairwise
d <- abs(means[c(1,1,1,2,2,3)] - means[c(2,3,4,3,4,4)]) / sqrt(((5-1)*vs[c(1,1,1,2,2,3)] + (5-1)*vs[c(2,3,4,3,4,4)])/ (5+5))
names(d) <- c("1-2", "1-3", "1-4", "2-3", "2-4", "3-4")
require(pwr)
# with 5 samples, we have the power to detect effect size f = 0.835
# i.e. with only 5 samples, we need a large effect to detect
pwr::pwr.anova.test(k = 4, n = 5, sig.level = 0.05, power = 0.80)
#>
#> Balanced one-way analysis of variance power calculation
#>
#> k = 4
#> n = 5
#> f = 0.8352722
#> sig.level = 0.05
#> power = 0.8
#>
#> NOTE: n is number in each group
# since we have a really large effect in the pilot for f = 1.2,
# we only need 3 per group to detect with 80% power
pwr::pwr.anova.test(k = 4, f = 1.2414, sig.level = 0.05, power = 0.80)
#>
#> Balanced one-way analysis of variance power calculation
#>
#> k = 4
#> n = 2.950833
#> f = 1.2414
#> sig.level = 0.05
#> power = 0.8
#>
#> NOTE: n is number in each group
Best Answer
Go download G*Power here and install.
In your case, there are a couple possibilities:
As paired sample t-test
Set it to
t test,Means: Difference from constant,Post hoc. The punch in the Effect size estimate, alpha and sample size. (You may click on "Determine" to find out how the effect sizes are calculated.) Power is 0.8856.As Wilcoxon singed-rank
Following similar steps, change the analysis to
Means: Wilcoxon signed-rank test (matched pairs), you can also estimate the power for Wilcoxon. It's about 0.8703.The host site also has documentation and tutorials. Feel free to read up if you have other new scenarios.
For t-test, it's more conservative to go for 2-tailed because your mean delta can be bigger or smaller than equality (aka 0).