Solved – 2×2 ANOVA, no main/interaction effects, post-hoc cell mean comparisons necessary/allowed

anovapost-hocr

I have a 2×2 between-subject design with unequal cell sizes. I ran an ANOVA with type I sums of squares to account for the unequal cell sizes.

Neither of the two main effects nor the interaction effects were significant (all ps > 0.1).

Cell means that were subjected to ANOVA

With an overall of 2×2 = 4 cells/groups there are obviously 6 possible pairwise comparisons, i.e. 6 individual differences between cell means that might be statistically significant. Can I conclude from the non-significant main and interaction effects that none of these pairwise comparisons are statistically significant?

As the plot suggests it would be particularly interesting to examine the difference between the two means on the left-hand side, i.e gain-absent vs. gain-present.

Would it be legitimate/necessary to run a Tukey-Kramer test (due to unequal cell sizes) to test this? If so, how do I do this in R? (Here I'm assuming that the standard TukeyHSD() function would be invalid due to unequal cell sizes)

Best Answer

No, you cannot conclude that none of the comparisons are significant. If the figure represents your data then I'm guessing that you're going after present higher than absent in gain but not in loss. If that's the case be very cautious because that's exactly what your interaction tested. And, a non significant interaction already told you that the difference between significant and not significant was not, itself, significant.

You might even only find gain:present higher than loss:absent, but what would that mean? Are all of tests even sensible or interpretable?

In short, before venturing forth with more tests seriously consider what they mean. I suggest that if you really consider the ANOVA result it has already tested any meaningful questions you'll have about the data. If you decide to make the comparisons anyway then answers to this question address yours as well.

Related Question