The Wilcoxon test does not test for equality of means, rather it tests
$$H_0: P(X_a > X_b) = 0.5$$
namely that a randomly drawn observation of group a has 50% chance of being larger than a randomly drawn observation from group b. Only if you see location-shift (i.e. distributions in both groups have the same shape but different mean (location)) than you can formulate your conclusion in terms of means. With the P-value you have you can reject $H_0$ on the 5% significance level. The $W$ is the Wilcoxon test statistic and is, as the name says, the sum of the ranks in one of both groups. You could enumerate the exact distribution of $W$ under $H_0$ by typing
wilcox.test(raj_reps.a, raj_reps.b, exact=TRUE)
This is very computer-intensive, so usually for such large sample sizes the asymptotic normality of $W$ is used for calculating the p-value, see ?wilcox.test
There are two obstacles to doing a Wilcoxon signed-rank test: (a) You have only 13 non-zero differences among 21. The 0 differences provide no evidence that Q1 and Q2 differ. You may hate to 'discard' these differences, but they were never really there. (b) There are many ties among the non-zero differences; only six unique differences among 13. Because the Wilcoxon test is based on ranks, the existence of
so many ties makes it difficult to find a P-value. I simply don't think a Wilcoxon signed rank test is useful for analyzing your data.
In principle, a simulated permutation test can obtain a reasonable approximation
of the P-value. However, that does not help find a significant difference for your data. P-values of permutation tests (one or two-sided) on your data are
much too big to lead to rejection.
The root difficulty is that 4 of the 13 nonzero differences are positive and 9 are negative, and that is too well balanced an outcome to lead to rejection. (If a fair coin is flipped 13 times, there
is better than 1 chance in 4 that it will show Heads either 4 or fewer times or 9 or more times.)
The bottom line is that your data are consistent with the null hypothesis that
Q1 and Q2 do not differ.
Addendum: Here is one possible permutation test for your data.
First, a paired t test gives test statistic $T = -0.8495$ with P-value $0.4057.$
t.test(d)
One Sample t-test
data: d
t = -0.8495, df = 20, p-value = 0.4057
alternative hypothesis: true mean is not equal to 0
95 percent confidence interval:
-0.5759201 0.2425867
sample estimates:
mean of x
-0.1666667
The legendary robustness of the t test notwithstanding, a t test may not be appropriate for your data because they are so discrete and because they
fail a Shapiro-Wilk test of normality with P-value $0.02569.$
shapiro.test(d)
Shapiro-Wilk normality test
data: d
W = 0.89302, p-value = 0.02569
The usefulness of the t statistic as a measure of any shift from Q1 to Q2 is
not in question. However, because the data are not normal, it is doubtful
whether the distribution of the t statistic is Student's t distribution with 20 degrees of freedom.
A permutation test is based on the idea that if there is no shift in values from Q1 to Q2, we could change the signs of the differences without harm. If we
change these signs at random many times, and compute the t statistic for each
such 'permuted' sample, then we can approximate the 'permutation distribution' of the t statistic, and use that distribution to get a reliable P-value.
There
are too many possible permutations to consider them all by combinatorial methods, but simulating many cases gives a serviceable result. The result will be
slightly different on each run, but with 100,000 iterations not enough
different to affect the conclusion whether to reject the null hypothesis.
(Results from a program in R are shown below.)
The run below gave P-value 0.49, which is very much larger than 5%, so we
cannot reject the null hypothesis. [The P-value is not enormously different from the P-value of the t test, but enough different that doing the permutation test was worthwhile.]
d = c(-.5, -2, 0, 2.5, -.5, .5, 0, -.5, -.5, -1, 0, -1, 0 ,0, 1, 0, 0, .5, -1, 0, -1)
n = length(d)
set.seed(706); m = 10^5
t.obs = t.test(d)$stat; pv.obs = t.test(d)$p.val
t.prm = replicate(m, t.test(d*sample(c(-1,1), n, rep=T))$stat)
p.val = mean(abs(t.prm) >= abs(t.obs)); p.val
[1] 0.48816
The permutation distribution of the t statistic is highly discrete, but
when plotted with only a few histogram bars it is seen not to differ greatly
from Student's t distribution with 20 degrees of freedom, the
density of which is shown in the figure below.
Reference: Eudey, et al. (2010) provides an
elementary introduction to permutation tests; Sect. 3 deals with paired
designs.
Best Answer
I usually turn to simulation for power calculations for the Wilcoxon sign-rank test. I have my own function I use for this. Use it at your own risk, as I don't know that anyone has ever validated it.
You can read the function directly using
or install the package (I haven't been actively developing it for a couple years) using:
In order to make it work, you'll need to estimate distributions from each of the groups in your sample. In the example below, I've assumed one group follows a Poisson distribution with a mean of 2.1, and the other follows a Poisson distribution with a mean of 3.53. I've also assumed equal sample sizes. This yields an estimate power of 0.444.