Solved – similarities between t-test ,welch test and mann whitney test

nonparametrict-testwilcoxon-mann-whitney-test

I am newbie to statistics field.
I find it is quite hard to understand this few tests even though they are common tools for testing hypothesis.

As i am referring this paper actually "should we always choose a nonparametric test when comparing two apparently nonnormal distribution?" by Skovlund and Fenstad.

This paper is to discuss which tools is the most appropriate to be use in different condition.(such as equality of variance or equal sample size).

Therefore, can i state that three of them is to test the equality of mean?

Best Answer

You are indeed correct that they are testing for the equality of the mean.

But you might be more interested in what makes them different from one another and what criteria should be meant to select which test. The tests can be ranked in generality. (Most general to least general)

The Mann Whitney test is the most general with the fewest assumptions, it is a nonparametric test, meaning that we do not need any distributional assumptions about the probability distribution that the data comes from.

The Welch test adds the assumption that the two groups comes from a normal distribution, but the variance in the two groups can be different.

The Student t-test adds yet another assumption, that the variance should be equal in the two groups.

But what do these assumptions imply? What effect does this have on our analysis?

The following is a small example from a built in dataset in R called mtcars. It has data about different types of cars and variables measured from these cars. If you use R you can use the summary command to get info about the data set

summary(mtcars)

Now we have variable called mpg, which is the mileage and am, which is 0 if the car is manual and 1 if the car is automatic. We want to test if there is a significant difference between the mileage of cars depending on whether they are automatic or manual.

We can plot the data to get some intuition about the answer:

plot(mtcars$am,mtcars$mpg)
title('mpg vs am')

plot of mpg vs am

We can see that the mileage of the automatic cars seems to be higher, but we would like to do a statistical test to conform this.

Starting with the Student t-test we get:

> t.test(mpg~am,data=mtcars,var.equal=TRUE)

    Two Sample t-test

data:  mpg by am
t = -4.1061, df = 30, p-value = 0.000285
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -10.84837  -3.64151
sample estimates:
mean in group 0 mean in group 1 
       17.14737        24.39231

And the Welch test:

> t.test(mpg~am,data=mtcars)

    Welch Two Sample t-test

data:  mpg by am
t = -3.7671, df = 18.332, p-value = 0.001374
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -11.280194  -3.209684
sample estimates:
mean in group 0 mean in group 1 
       17.14737        24.39231

And finally the Mann Whitney test:

> wilcox.test(mpg~am,data=mtcars)

    Wilcoxon rank sum test with continuity correction

data:  mpg by am
W = 42, p-value = 0.001871
alternative hypothesis: true location shift is not equal to 0

The $p$ values are increasing for these tests, i.e.

  • t-test has $p$-value = 0.000285

  • Welch test has $p$-value = 0.001374

  • Mann Whitney test has $p$-value = 0.001871

So when you relax on your assumptions you become less certain of the outcome in some sense. If your $\alpha$ value of the test would be 0.1\% (Which is rarely used except when doing multiple testing/comparisons or maybe in pharmaceutical experiments), then you would reject the null hypothesis using the t-test but not with the other tests.

If you know that the data comes from a normal distribution and the two groups have equal variance, this is the correct thing to do. People often tend to assume that the data comes from a normal distribution, but if you want to be conservative the Mann Whitney test is more appropriate.

Related Question