I have these groups where the values are responses to a 10-point Likert item:
g1 <- c(10,9,10,9,10,8,9)
g2 <- c(4,9,4,9,8,8,8)
g3 <- c(9,7,9,4,8,9,10)
Therefore I used Kruskal-Wallis to determine any differences between responses in the groups, and the result was:
Kruskal-Wallis chi-squared = 5.9554, df = 2, p-value = 0.05091
However, if I run an exact Mann-Whitney test between groups g1 and g2 I get:
Exact Wilcoxon Mann-Whitney Rank Sum Test (using coin::wilcox_test)
Z = 2.3939, p-value = 0.02797
which returns a significant difference at alpha = 0.05.
Which test should I choose, and why?
Best Answer
I agree with Michael Chernick's answer, but think that it can be made a little stronger. Ignore the 0.05 cutoff in most circumstances. It is only relevant to the Neyman-Pearson approach which is largely irrelevant to the inferential use of statistics in many areas of science.
Both tests indicate that your data contains moderate evidence against the null hypothesis. Consider that evidence in light of whatever you know about the system and the consequences that follow from decisions (or indecision) about the state of the real world. Argue a reasoned case and proceed in a manner that acknowledges the possibility of subsequent re-evaluation.
I explain more in this paper: http://www.ncbi.nlm.nih.gov/pubmed/22394284
[Addendum added Nov 2019: I have a new reference that explains the issues in more detail https://arxiv.org/abs/1910.02042v1 ]