Solved – Are pairwise Wilcoxon tests a valid non-parametric alternative to Tukey’s HSD test

kruskal-wallis test”nonparametricpaired-datat-testtukey-hsd-test

I am trying to identify significant differences between groups from normal and non-normal distributions. When the distribution is normal the workflow is pretty straightforward: ANOVA and after that a Tukey's HSD test in order to identify the groups which are responsible of the significant variation. When the distribution is non-normal I apply a Kruskal-Wallis test and after that a pairwise Wilcoxon test (R function pairwise.wilcox.test()) to identify the groups. I know that Wilcoxon test is the non-parametric version of the t test, and is not corresponding to the Tukey's HSD test, but I want to know if what I am doing is wrong and which is the main difference between the outcomes of a t-test and a Tukey's test since they both give differences between groups.

Best Answer

No, it is not a valid nonparametric alternative.

The rank sum test (either original Wilcoxon flavor, or New Improved Mann-Whitney $U$ varieties):

  1. ignore the rankings used by the Kruskal-Wallis test, and
  2. do not employ pooled variance for the pairwise tests.

See, for example, Kruskal-Wallis Test and Mann-Whitney U Test. (Also the pairwise.wilcox.test seems not to have the ties adjustments that these tests do.)

The nonparametric pairwise multiple comparisons tests you are likely looking for are Dunn's test, the Conover-Iman test, or the Dwass-Steel-Crichtlow-Fligner test. I have made packages that perform Dunn's test (with options for controlling the FWER or FDR) freely available I have implemented Dunn's test for Stata and for R, and have implemented the Conover-Iman test for Stata and for R.


References
Conover, W. J. and Iman, R. L. (1979). On multiple-comparisons procedures. Technical Report LA-7677-MS, Los Alamos Scientific Laboratory.

Crichtlow, D. E. and Fligner, M. A. (1991). On distribution-free multiple comparisons in the one-way analysis of variance. Communications in Statistics—Theory and Methods, 20(1):127.

Dunn, O. J. (1964). Multiple comparisons using rank sums. Technometrics, 6(3):241–252.