Solved – Determining how much data is needed for statistical significance of a percentage difference between two points

statistical significance

Imagining the following situation where A/B are independent results:

  • A is a 51% result, given N datapoints
  • B is a 50% result, given M datapoints

how many data points (ie N and M) do you need to have confidence that the difference between A/B is statistically significant?

Best Answer

There are various sample size calculators you can use online and in various software for a two-sample proportions test.

Here's the calculation using Stata that suggests that you would need N=M=39,240 for a two-sided alternative test at conventional levels:

. power twoproportions .50 .51

Performing iteration ...

Estimated sample sizes for a two-sample proportions test
Pearson's chi-squared test 
Ho: p2 = p1  versus  Ha: p2 != p1

Study parameters:

        alpha =    0.0500
        power =    0.8000
        delta =    0.0100  (difference)
           p1 =    0.5000
           p2 =    0.5100

Estimated sample sizes:

            N =    78,480
  N per group =    39,240
Related Question