Solved – Confidence interval does not include 0, but Wilcoxon test is sgnificant

meannonparametricrepeated measuresspss

I've created an error plot with CI 95% to visualize the difference (post test – pre test) between achieved scores.

error bars

n = 64

Judging from the confidence intervals one could conclude that there's is no significant improvement between the pre and post test scores (the CI does include 0).

When I use a paired t-test for the pre and post scores in combination, none of them return significant as expected.

But since the differences of the scores are not normally distributed for Total Score Change (p < .001) and Sub Score Change I (p < .05) according to Shapiro-Wilk, i've use the paired Wilcoxon test, which turns out significant for Sub Score Change I (p < .05).

So which test should I believe and why?

Best Answer

You need to use a point estimate (pseudo median) and confidence interval that is consistent with the Wilcoxon signed-rank test. Here is an example in R, where there are no ties in absolute values of diff so that exact calculations are available (otherwise use the R coin package).

diff <- c(-3.1, -2.2, 1.3, 2.4, 3.5, 4.6, 5.7)
wilcox.test(diff, exact=TRUE, conf.int=TRUE, conf.level=0.95)

Wilcoxon signed rank test

data:  diff
V = 22, p-value = 0.2188
alternative hypothesis: true location is not equal to 0
95 percent confidence interval:
 -2.2  4.6
sample estimates:
(pseudo)median 
           1.8

Note that the pre-post design does not allow cause and effect type of inferences.

Related Question