Solved – Possibility to run multiple chi-square tests on 2×5 table

chi-squared-testhypothesis testingmultiple-comparisons

I'm analyzing data in which I have a dependent variable with two possible outcomes (yes or no) and an independent variable with five emotion conditions, neutral, angry, sad, ashamed, and afraid. The 2×5 table is not statistically significant with a chi-square test, (probably) because the outcomes for some conditions are very close together (e.g., the distribution of yes/no is very similar for ashamed and afraid). However, a pairwise comparison between only certain conditions (e.g., comparing neutral to sadness) does give significant chi-square results.

Is it allowed to do pairwise comparisons with chi-square when overall the test is insignificant? And are post hoc corrections (e.g. Bonferroni) necessary in this case?

Best Answer

This is not really different from the standard case of multiple groups with a continuous variable. If you have an a-priori hypothesis that $B < E$, and don't care about the relationships amongst the other groupings, you can simply run a t-test on those two. In your case, the response variable is binary, so you could simply run a $2\times2$ chi-squared test. If you didn't have such an a-priori hypothesis, you don't want to look at the data and notice that $B$ and $E$ are the furthest apart and just test those. In general, it is best to see if there are any differences amongst the groups with a single (ANOVA or chi-squared) test first. That approach provides some protection against type I error inflation. If you don't have an a-priori hypothesis and you want to skip the omnibus test, you definitely need some strategy to hold familywise type I error rates at an acceptable level (e.g., Bonferroni would be one possibility).

Given more information about your situation, what you want is some form of Dunnett's test. The traditional version of Dunnett's test is for a continuous response variable. In your case, you need to do this with a logistic regression model. This pdf provides information on how to do that. Note that it is written for SAS, but you may be able to adapt the procedures for your chosen software.

Related Question