Solved – Level of Bonferroni Correction for Post-hoc Wilcoxon Signed Rank

bonferronipost-hocwilcoxon-signed-rank

I have conducted a study with a within subject design where all participants undertook 4 conditions. I performed a Friedman Test on the ordinal data. I then used Wilcoxon Signed Rank test as a post-hoc test.

I wanted to ask about the Bonferroni correction. These 4 conditions result in 6 possible pairwise comparisons for the Wilcoxon, so a Bonferroni correction would result in p = 0.05/6.

But for my analysis, I would just consider 4 of the 6 possible cases. Would this mean, that I just have to correct my p for the 4 comparisons I am interested in? I would also like to know in terms of how I should report this correction in the paper.

Best Answer

General considerations (ignoring the specifics of your tests)

If you only do four comparisons you only do four comparisons, as long as that choice was made before you saw any data.

The number of possible comparisons that you might have been able to do but didn't wouldn't be relevant to that count.

However, one potential issue could be that it may turn out that it seems as if you choose your comparisons post-hoc (partiularly if it works out well for what you're looking for). You'd need a very clear and solid up-front justification for looking only for what you're looking for.

Specific considerations

  1. Since you're actually doing this as a post-hoc, and an overall rejection may be due to any of the things being different, your audience will expect you to do all six. [In effect a post hoc is for assigning blame for the overall test. If you're not interested in the overall result except as it relates to the four comparisons you're interested in, it might perhaps be best to just focus on those.]

  2. The Friedman and the signed rank test don't correspond; they're sensitive to different things. The sign test would be closer to a two-sample equivalent of the Friedman. However, I'm not making the claim that plain sign tests are the best possible choice of a post hoc for a Friedman test; that might, for example, be better accomplished by conditioning the pairwise comparison on the actual ranks assigned to those two groups from the Friedman (I presume there's a Dunn-type post hoc for it that does this already).

Related Question