Solved – Power analysis on chi-squared test with low cell counts

assumptionschi-squared-teststatistical-power

I am hoping to perform a chi-square test of independence on data in a 2×2 contingency table with the following values:

Group A: 627 successes, 28 failures
Group B:  59 successes,  2 failures

I understand that this is a small collection of observations, and I wish to determine the minimum number of observations required for a chi-squared test at 0.05 significance, 0.8 power. However, it seems that the normal approximation of the binomial distribution that is generally used for power analysis among comparisons of proportions is not appropriate here for at least two reasons:

  1. p(success) is nowhere near 0.5 for the populations AND
  2. the expected number for Group B Failures, (2+59)*(2+28)/716 = 2.56, is below the guideline of 5 for all cells in chi-square power analysis.

Any constructive guidance on how to perform this test of independence (or reference to other questions I may have overlooked) is greatly appreciated.

Best Answer

@whuber is right that only the expected counts matter. Reading some of the threads returned by his search may help you; I discuss the issue here: For chi-square on any 2 by X contingency table, should no more than 20% of the cells be less than 5? You can also see the point made under "Expected cell count" on the Wikipedia page.

Note that the expected cell count is the probability times the N. Your effects are so small that huge N will be required. Here are screen shots of G*Power using your ratio of n's, or equal n's:

enter image description here

enter image description here

Your lowest probability is $3.3\%$, and your lowest count is $6187$, meaning that the expected count would be $204\gg 5$. Thus, @rvl is right: you won't have to worry about that assumption. (You will have to worry about needing so much data to have reasonable power, I suspect.)

Related Question