Solved – Power of chi-square test for goodness-of-fit as function of sample size

chi-squared-testgoodness of fitrocstatistical-power

I would like to calculate the power of a Chi-Square test for goodness-of-fit as a function of sample size for a specified alpha-value (say 0.01). Specifically, I am referring to power as the probability that the test will correctly reject the null hypothesis. In most places I look, I can find only vague references that this can be done and/or am referred to software. It seems to me that this should not be difficult. The specific question I would like to be able to answer (without using some sort of opaque software) follows:

Given an $\alpha$ value, for what sample size, $n$, can I expect a particular power, $\beta$?

Even better, how might I be able to calculate an ROC?

Best Answer

Does R fall into your idea of opaque software? If you are interested in this sort of calculation I would strongly recommend you use a stats package of some sort - and probably R in particular. In R, package pwr provides the function pwr.chisq.test which answers your question for you.

It isn't quite as simple as saying "for each sample size, what is the power of my test?" because as well as sample size, there is the question of the size of the effect in the underlying population you are inferring to. eg if there is a massive effect, then even a very small sample has a high power. As the effect gets smaller, you need a bigger sample size for the same power.

The documentation for pwr.chisq.test refers to Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale,NJ: Lawrence Erlbaum.

Also, a quick google search comes up with this reference (lecture 25), which shows that under the alternative hypothesis the test statistic has (asymptotically) a non-central Chi square distribution and provides a way to estimate the non-centrality parameter for a given alternative hypothesis.

Related Question