Solved – Comparing 5 proportions/Post-hoc tests after $\chi^2$

chi-squared-testcontingency tables

I have one parameter, that is, a behaviour happened or not. I have then the proportions of animals who performed the behaviour under different treatments. I want to compare if there are any difference between the treatments. Specifically if treament A differs from treatment B or C, etc. I have only GraphPad Prism and I have tried to run a $\chi^2$ but it didn't answer my question at all.

Different animals are being subjected to different treatments. There are 5 groups (1 control group, 4 groups on different doses of drug). “$\chi^2$ for trend” only gave me a p value. But I don't know if treatment A differs from B or C. I need something like a post-hoc test to compare different treatments.

The table is something like that, the numbers mean numbers of animals who performed the behaviour (escape reaction)

            Escape reaction    No escape reaction
Control:                  0                     5 
Treatment A:              4                     3
Treatment B:              3                     3
…

Best Answer

Your question is still somewhat ambiguous, is it “A differs from B or C, etc.” (suggesting you want to tests many comparisons, at least informally) or “A differs from B or C” (one specific test, which does however begs the question of why you had 5 conditions in the first place if only this comparison is important)?

In any case, you can always consider some subset of conditions or collapse two conditions together and still analyze the resulting contingency table with a $\chi^2$ test of independence. There are systematic ways to define these subtests (look up “partitioning chi squared”), which might in fact be implemented in the software you are using, saving you the trouble of defining new contingency tables.

If you are doing many such tests and the comparisons were not defined beforehand, you might want to first do an overall test of independence on the 5x2 contingency table and only proceed further if the overall null hypothesis is rejected. Alternatively, you might consider some adjustment for multiple comparisons.

Yet, another way to follow-up a $\chi^2$ test of independence, especially when you don't have specific comparisons in mind, is to look at standardized Pearson residuals.

Partitioning and looking at residuals are somewhat analogous to post-hoc tests for the $\chi^2$ test. Both of these are discussed in Alan Agresti's book Categorical Data Analysis.

PS: Re-reading your edit, I see that the different conditions are different doses of the same drug. The techniques discussed above do not take the nature of the manipulation into account, it might be worthwhile to model it directly, perhaps with a logistic regression?

Lastly, if this is all too much or does not exactly apply to your situation, you could also consider a “minimalist” approach:

  1. $\chi^2$ test of independence on the whole contingency table, as a rough check that you have something else than noise/sampling variation
  2. Plot/direct interpretation of the pattern of proportions, perhaps with a confidence interval around each proportion.

This is not perfect and might not fly in some “test-centered” disciplines but it seems perfectly reasonable to me.

Related Question