Solved – How to interpret a significant one-way repeated measures ANOVA with non-significant pairwise, bonferroni adjusted, comparisons

anovapost-hocrepeated measures

I have a dataset with one factor consisting of four levels. I've run a one-way repeated measures ANOVA on the data, which came out significant. However, for the follow-up pairwise, bonferroni adjusted, comparisons, none of them came out significant. (Why this is possible have been covered in several threads before, for example this one.)

Now, I don't really know how to interpret my data. Can I conclude that there at least is a difference between the two levels in my data set with the largest difference, because the ANOVA came out significant, or will I have to reject this conclusion because my pairwise comparisons all came out non-significant?

Note that it might be the case that I, in my specific situation, would be able to find a post-hoc test that came out significant, but my question is more directed towards the general question of how to interpret the analysis results in a situation where a significant post-hoc couldn't be acquired.

Also, this question is somewhat related to mine.

Best Answer

The reason that the ANOVA rejects could be due to violations in other assumptions such as homogeneity of variances/sphericity. (See assumption #5 here:https://statistics.laerd.com/spss-tutorials/one-way-anova-repeated-measures-using-spss-statistics.php)

If for example sphericity is violated, then you can conclude that the significant result from ANOVA is likely due to violations of the ANOVA assumptions as opposed to a significant difference between two levels.

If the important ANOVA assumptions are met and you still run into this issue, then you can conclude that there is a possible difference between at least two of the levels, but that the significant ANOVA may be a false positive.

This page explains multiple comparisons: http://www.biostathandbook.com/multiplecomparisons.html

Another possible conclusion/explanation for your particular case using Bonferroni adjustments is that the Bonferroni method is very conservative so even if a difference between the levels exist, multiple comparisons with Bonferroni adjustments may not have been able to detect it.