Solved – Has anyone used the Marascuilo procedure for comparing multiple proportions

chi-squared-testmultiple-comparisons

The Marascuilo procedure as described here seems to be a test that addresses the issue of multiple comparisons for proportions when you want to test which specific proportions are different from each other after rejecting the null in an overall chi-square test.

However, I am not very familiar with this test. So, my questions:

  1. What nuances (if any) should I worry about when using this test?

  2. I know of at least two other approaches (see below) to address the same issue. Which test is the 'better' approach?:

Best Answer

Just a partial answer because I've never heard of this method. From what I read in the link you provided, it seems to be a single-step procedure (much like Bonferroni, except we rework the test statistics instead of the p-value) which is likely to be too conservative.

In R, there is a function pairwise.prop.test() which allows any correction for multiple comparisons (single-step or step-down FWER methods or FDR-based), but it is quit what you already suggested (although Bonferroni is by far too conservative, but still very used in practice). A resampling approach, using permutation, might be interesting too. The coin R package provides a well-established testing framework in this respect, see §5 of Implementing a Class of Permutation Tests: The coin Package, but I never had to deal with permutation tests on categorical data in a post-hoc way.

About the analysis of subdivided contingency tables, I generally consider specific associations as a guide to develop additional hypotheses (as for any unplanned comparisons), but this is another question. I generally just use visualization tools, like mosaicplot from Michael Friendly, Pearson's residuals, and if I seek to explain specific patterns of association I use log-linear models.