Solved – Why run a post hoc wilcoxon signed rank test after Friedman test, why not just skip to Wilcoxon sign test instead

friedman testmultiple-comparisonsnonparametricpost-hocwilcoxon-signed-rank

I understand that one is required to run post hoc tests after the Friedman test. For example, while the Friedman test may find a statistically significant change amongst 3 treatments, one may follow with Wilcoxon signed rank tests to find where that change is (amongst which 2 treatments).

Why not just run multiple Wilcoxon signed rank tests (and divide a/3) instead? Why the Friedman?

Best Answer

The short answer is that needless multiple pairwise testing will taint your inference. That is, if you conduct multiple pairwise comparisons, the probability of falsely rejecting the NULL of in at least one of these tests increases as the number of pairwise comparisons increases.

This is the multiple testing problem that is typically introduced in a stat class to motivate Analysis of variance. Notice that a pairwise testing procedure assumes that each test is independent of one another. This assumption cannot be true since in comparing 3 groups, the same groups are repeatedly used across tests.

Often the Bonferroni method is discussed as a means of controlling for this distortion of the type 1 error rate, where the desired p-value is multiplied by as many tests as are conducted.

By first conducting a test that is designed to compare multiple parameter estimates, the correct inference can be made (as long as the assumptions for that test hold). If this test rejects the NULL of equality of parameters, then the post hoc methods can be employed to determine which parameter(s) is(are) different.