Solved – Interpreting contradicting results T-Test vs. Mann-Whitney test (2 independent samples)

meanmediant-testwilcoxon-mann-whitney-test

I want to test if two indepent experimental groups are sign. different in terms of one outcome variable, but face the following questions:

Basic setup

  • 2 groups with 200 observations each
  • Outcome variable:
    • continuous
    • ranging between 0 and 600
    • very peaked: more than 60% of values are equal to 300
    • Group 1: mean = 221.9807, median = 300
    • Group 2: mean = 239.6396, median = 300

Result of statistical tests

Although the outcome varible violates the normality assumption, I orginally used a two-sample independent t-test based on my large group sizes and the central-limit theorem:

Result of T-test: no signifcant difference (p=0.166).

However, to be on the safe side I also used a nonparmetric test, i.e. the Mann-Whitney U test:

Result: of Mann-Whitney U test significant difference (p=0.023).

Questions

  • My interpretation: Mean outcome not sig. different, but distribution of outcome varible differs between groups. Is that correct?
  • Could it be a problem that the vast majority of my values are equal to 300? Does the MW test ignore values that are equal to the median?

Best Answer

The original Mann-Whitney test assumes continuous distributions.

If there are many observations and few ties, a normal approximation with a correctly-calculated variance is sufficient.

To deal with heavy ties, the Mann-Whitney test needs to properly deal with the effect those ties have on the distribution of ranks under the null; in some cases the effect can be substantial.

What happens in that situation varies from package to package - some packages don't handle heavy ties well.

The p-value from the t-test may well be suspect.

I'd be inclined to perform a permutation test on the actual set of ranks, or if primary interest focuses on testing for a difference in means, perhaps a permutation test based on the means; this way I don't have to rely on either the t-statistic having a distribution close to the t-distribution under the null, nor on the Mann-Whitney correctly dealing with heavy ties.

Related Question