Solved – Post-hoc test for chi-square goodness-of-fit test

chi-squared-testgoodness of fithypothesis testingpost-hoc

I'm conducting a chi-square goodness-of-fit (GOF) test with three categories and specifically want to test the null that the population proportions in each category are equal (i.e., the proportion is 1/3 in each group):

                OBSERVED DATA
Group 1     Group 2     Group 3     Total
  686              928            1012        2626

Thus, for this GOF test, the expected counts are 2626(1/3) = 875.333 and the test yields a highly-significant p-value of < 0.0001.

Now, it's obvious Group 1 is significantly different from 2 and 3, and it's unlikely that 2 and 3 are significantly different. However, if I did want to test all of these formally and be able to provide a p-value for each case, what would be the appropriate method?

I've searched all over online and it seems there are differing opinions, but with no formal documentation. I'm wondering if there is a text or peer-reviewed paper that addresses this.

What seems reasonable to me is, in light of the significant overall test, to do z-tests for the difference in each pair of proportions, possibly with a correction to the $\alpha$ value (maybe Bonferroni, e.g.).

Best Answer

To my surprise a couple of searches didn't seem to turn up prior discussion of post hoc for goodness of fit; I expect there's probably one here somewhere, but since I can't locate it easily, I think it's reasonable to turn my comments into an answer, so that people can at least find this one using the same search terms I just used.

The pairwise comparisons you seek to do (conditional on only comparing the two groups involved) are sensible.

This amounts to taking group pairs and testing whether the proportion in one of the groups differs from 1/2 (a one-sample proportions test). This - as you suggest - can be done as a z-test (though binomial test and chi-square goodness of fit would also work).

Many of the usual approaches to dealing with the overall type I error rate should work here (including Bonferroni -- along with the usual issues that can come with it).