Solved – Difference Estimate & Confidence Interval for Paired Mann-Whitney Test

confidence intervalhypothesis testingwilcoxon-mann-whitney-test

I have two paired samples and I am running a paired Mann-Whitney test to determine whether they are different. In Matlab, this takes the form of p = signrank(x,y), which tests the null hypothesis that x-y comes from a population of median zero.

Unfortunately, Matlab doesn't give difference estimates for x-y or confidence intervals for this estimate.

In the past, to estimate the shift estimate for nonpaired samples, I used the Hodges-Lehmann estimate (see Difference of 'centers' of 2 non-normal samples with Mann-Whitney test & http://en.wikipedia.org/wiki/Hodges%E2%80%93Lehmann_estimator). This is essentially the median of all pairwise differences.

Here, since the samples are paired, I imagine that the HL estimate is incorrect. Intuitively, I was thinking of using the median of all paired differences.

In either case, my question is how to get the difference estimate for x-y and the confidence interval for that estimate.

Below is a picture of my data:

The top histogram shows both samples (correct & incorrect), the bottom histogram shows the pairwise differences. A boxplot of the data is on the right. The estimated p-Value for the paired MW test is above & is <.05. Thus the data is significantly difference with correct > incorrect (even though it admittedly is hard to see on the histogram. Any suggestions on how to make it more visualisable are greatly appreciated!)

data

Best Answer

One can construct a confidence interval for the median difference using Walsh averages. It does assume that the distribution of the differences is symmetric, but it seems to be OK here. See this document for an explanation of the procedure. It appears that somebody even wrote a Matlab function for this calculation.

EDIT: implementation in Matlab The psignrank function in R calculates the cumulative distribution of the Sign-rank test statistic. If Matlab does not have that function, then you probably can't easily calculate an exact confidence interval. However you can calculate an approximate interval as described here. I think that is the calculation attempted at the end of the Matlab code.

Related Question