Suppose in group 1 there are $n$ people and in group 2 there are $n$ people. If we conduct a 2 sample t test versus a paired t test, would we use $2n$ for power calculations in the two sample t test and $n$ as the sample size for the paired t test?
Solved – How does power analysis differ between paired sample and independent groups t-test
statistical-powert-test
Related Solutions
Yes, this is possible and even fairly easy, but additional information is required. Specifically, we have to make an assumption about what the correlation between the observations from each pair are.
The effect size as a difference in standard deviation units is usually referred to as $d$. We can apply a correction factor to $d$ to incorporate the information about the aforementioned correlation, and then we can use our standard power formulae with this corrected $d$ (making sure to also mind the change in degrees of freedom associated with moving to the paired design) to compute power. The corrected $d$ is $$ d_o = \frac{d}{\sqrt{1-r}}, $$ where $r$ is the correlation. I have called this $d_o$ because this is sometimes referred to as the "operative effect size."
Here is a little R routine that computes a table of minimum number of PAIRS as a function of the assumed correlation and the desired power level, with $d=2$ assumed.
library(pwr) # package for pwr.t.test() function
# may need to install first with install.packages()
# define a function to get the minimum number of pairs
# for a given correlation and desired power level
getN <- function(r,p){
unlist(mapply(pwr.t.test, d=2/sqrt(1-r), power=p,
MoreArgs=list(n=NULL, sig.level=.05, type="paired"))["n",])
}
# apply this function to all combinations of the parameters below
tab <- outer(seq(0,.95,.05), c(.7,.8,.9,.95,.99,.999), "getN")
dimnames(tab) <- list("Correlation"=seq(0,.95,.05),
"DesiredPower"=c(.7,.8,.9,.95,.99,.999))
tab
Which returns the following:
DesiredPower
Correlation 0.7 0.8 0.9 0.95 0.99 0.999
0 3.767546 4.220731 4.912411 5.544223 6.888820 8.656788
0.05 3.691858 4.126240 4.787326 5.389850 6.669683 8.350091
0.1 3.615930 4.031562 4.662220 5.235637 6.451021 8.044096
0.15 3.539645 3.936653 4.537050 5.081483 6.232774 7.738792
0.2 3.462940 3.841433 4.411750 4.927417 6.014903 7.434270
0.25 3.385708 3.745774 4.286234 4.773338 5.797404 7.130529
0.3 3.307922 3.649640 4.160447 4.619143 5.580267 6.827580
0.35 3.229382 3.552889 4.034209 4.464751 5.363362 6.525430
0.4 3.149970 3.455310 3.907393 4.309986 5.146613 6.224026
0.45 3.069435 3.356743 3.779777 4.154653 4.929824 5.923282
0.5 2.987581 3.256903 3.651065 3.998456 4.712773 5.623032
0.55 2.904079 3.155423 3.520841 3.841020 4.495111 5.323066
0.6 2.818472 3.051834 3.388672 3.681805 4.276260 5.022875
0.65 2.730145 2.945449 3.253781 3.520048 4.055501 4.721751
0.7 2.638237 2.835369 3.115118 3.354639 3.831565 4.418560
0.75 2.541442 2.720152 2.971074 3.183823 3.602697 4.111397
0.8 2.437713 2.597460 2.819127 3.004879 3.365682 3.796879
0.85 2.323340 2.463226 2.654597 2.812710 3.114890 3.468745
0.9 2.190677 2.309002 2.467897 2.596901 2.838233 3.113596
0.95 2.018024 2.110699 2.231866 2.327720 2.501567 2.692358
Note that $d=2$ is considered in many fields quite a large effect size, so the resulting minimum numbers of pairs are all quite low.
You can do sample size calculations for unequal sample sizes.
For example, you can decide the n's are in some ratio (such as in proportion to the populations perhaps).
It's then possible to do power calculations (at the least you can simulate to obtain the power under any particular set of circumstances, whether or not you are able to do the algebra).
The problem is that it's relatively inefficient at finding differences compared to the same total number of observations at equal sample sizes.
Imagine you had a total sample of $n=n_1 + n_2$, with equal variance in the population and close to equal sample variance, and that your choice was between a 50-50 split and a 90-10 split ($n_1 = 0.5n$ vs $n_1=0.9n$).
The two-sample t-statistic is:
$t = \frac{\bar {X}_1 - \bar{X}_2}{s_{\text{pooled}} \cdot \sqrt{\frac{1}{n_1}+\frac{1}{n_2}}}$
The impact of the sample size is in the term $1/{\sqrt{\frac{1}{n_1}+\frac{1}{n_2}}}$.
If you have the 50-50 split it's like having a 40% smaller standard deviation; at a given $n_1+n_2$ you can pick up a substantially smaller effect with the even split.
If the combined sample size is not an effective constraint, this calculation may pointless however. It matters in cases where every observation carries the same marginal cost, which is not always relevant.
Best Answer
In general, the way that input is specified depends somewhat on the equations or software that you use to calculate the statistical power. But yes, it would be normal to specify the sample size in paired-samples t-test as the sample size $n$, and for independent groups t-test it would be the sum of the two group sample sizes, which when equal would be $2n$.
If you take post-hoc power analysis in G-Power 3 for example, you specify the sample size for paired-samples and you specify the individual group sample sizes for independent groups t-test.
The big additional factor for power analysis in paired samples t-test is the correlation between the two levels of the repeated measures factor. Conditional on the null hypothesis being false, the size of the correlation is positively related to the amount of statistical power.