Solved – Reporting significance in Kruskal Wallis test

kruskal-wallis test”multiple-comparisons

I performed the Kruskal-Wallis one-way ANOVA by ranks on five groups of data using Dataplot software. The software corrects for ties and uses Conover's procedure to do pairwise multiple comparisons.

The result of the test is $H=5.62$ and $p=.229$ so the results are not significant ($p$ is not less than $.05$). However, the pairwise multiple comparisons show that the group 1 data significantly differs ($p<.05$) from the group 2 data.

I report that there are no significant differences in the five groups (because $p=.229$).

My question is this: I am using box plots to graphically represent the results. Should I indicate that the group 1 data and the group 2 data significantly differ even though the H-test says that there are no significant differences in the five groups of data?

Best Answer

I'm turning guest's comment into an answer so that this question is not marked unanswered.

No, not unless you're prepared to defend why the significance of this one pairwise comparison (out of 10 such possible comparisons) is greater than would be expected by chance alone.

Related Solutions

Solved – Statistical significance test: One way Anova and Kruskal-Wallis test

In general, you wouldn't necessarily expect one way ANOVA and the Kruskal-Wallis to be similar, sometimes they can give quite different p-values. See here for a little partial motivation for why you might expect a difference. [When samples are reasonably normal-looking and with means not too many standard errors apart, they often tend to give similar p-values. Outside that, they frequently don't.]

However, in this case the reason is more prosaic: Your Kruskal-Wallis p-value is wrong.

Here's a summary of results in R (details below).

                     p-value
Welch t-test:        0.001287
Equal-var. t-test:   8.552e-05
One way anova:       8.55e-05 
Wilcoxon test:       0.004847
Kruskal-Wallis:      0.003761

(Neither of the last two p-values are exact; if they were, you'd get the same p-value for the two-group comparison.)

Your problem is you're treating the second group's data as a factor (see the end of this answer).

Here's what I get in R with your data:

frh <- data.frame(group1 = c(103.56, 103.32, 103.32, 104.27, 103.56, 103.8),
                  group2 = c( 97.16,  97.16,  96.69,  98.58,  90.76,  97.64))

# strip chart:

enter image description here

# Welch t-test:
> with(frh,t.test(group1,group2))

    Welch Two Sample t-test

data:  group1 and group2
t = 6.3316, df = 5.163, p-value = 0.001287
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
  4.368147 10.245186
sample estimates:
mean of x mean of y 
103.63833  96.33167

$\,$

# equal-variance t-test:
> with(frh,t.test(group1,group2,var.equal=TRUE))

    Two Sample t-test

data:  group1 and group2
t = 6.3316, df = 10, p-value = 8.552e-05
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 4.735411 9.877922
sample estimates:
mean of x mean of y 
103.63833  96.33167

$\,$

#one way anova:
summary(aov(values~ind,stack(frh)))
            Df Sum Sq Mean Sq F value   Pr(>F)    
ind          1 160.16   160.2   40.09 8.55e-05 ***
Residuals   10  39.95     4.0

$\,$

# Wilcoxon-Mann-Whitney:
> with(frh,wilcox.test(group1,group2))

    Wilcoxon rank sum test with continuity correction

data:  group1 and group2
W = 36, p-value = 0.004847
alternative hypothesis: true location shift is not equal to 0

Warning message:
In wilcox.test.default(group1, group2) :
  cannot compute exact p-value with ties

$\,$

# Kruskal-Wallis test:
> kruskal.test(frh)

    Kruskal-Wallis rank sum test

data:  frh
Kruskal-Wallis chi-squared = 8.3958, df = 1, p-value = 0.003761

Those are all about as consistent with each other as I would expect on that data.

Now, here's how to get what you got for the Kruskal-Wallis:

with(frh,kruskal.test(group1,group2))

    Kruskal-Wallis rank sum test

data:  group1 and group2
Kruskal-Wallis chi-squared = 4.3939, df = 4, p-value = 0.3553

The problem is, if you're getting this, you're using it wrong. That's not how the function works - group2 is being treated as a factor defining different groups for data in group1.

So the main reason the Kruskal Wallis isn't giving you a roughly similar p-value to ANOVA is you didn't call it correctly.

Solved – Reporting Kruskal-Wallis Test Result with Pairwise Comparisons

Yes, keep the overall test and then write that you conducted pairwise tests. I would do something like this (but I'd change the writing to relate it more to the data)

"A Kruskal-Wallis test showed that at there was a significant difference of means (H = 18.047, p <0.001). I then conducted post hoc tests to test pairwise comparisons. I found that group A was significantly different to group B (p = 0.001) and group C (p = 0.002). Groups B and C were not significantly different (p = 0.23)."

That would be my 'template answer'. I'd alter it to fit your data.

Best Answer

Related Solutions

Solved – Statistical significance test: One way Anova and Kruskal-Wallis test

Solved – Reporting Kruskal-Wallis Test Result with Pairwise Comparisons

Related Question