I have 12 groups of datasets (mostly non-Normal, quite right-skewed), and I would like to see if there is a significant difference within each pair. (But the data itself is not a paired dataset, as in dataset #1 and dataset #2 that I am comparing are from independent populations.)
My data also has outliers (following the 1.5*IQR rule), so I understand that the t-test is not valid because (x bar) and s are not robust against outliers. The main alternative I see is to use the rank test Mann-Whitney U test. (Let me know if I am not on the right track.)
In Zar's Biostatistical Analysis (4th ed, pg. Ap100): "For the Mann-Whitney test involving n1 and/or n2 larger than those in this table, the normal approximation may be used." The table goes to n1=20, n2=40.
In Moore, McCabe, Craig's Introduction to the Practice of Statistics (6th ed., pg. 432): For sample sizes 15≤ n ≤39, "t procedures can be used except in the presence of outliers or strong skewness." For samples sizes ≥40, "t procedures can be used even for clearly skewed distributions."
Does this mean if n for one of my populations ≥20 (or ≥40?), I will have more accurate/powerful results using the 2-sample t-test (assuming unequal variance) instead? What if n1 ≤20 but n2 ≥20? If I need to use the Mann-Whitney U test for some, should I be using Mann-Whitney U test for all in order to compare the results? Are there other hypotheses tests I should be using instead?
For reference, the populations that I am comparing have counts:
n1 = 15, n2 = 20
n1 = 7, n2 = 27
n1 = 12, n2 = 11
n1 = 13, n2 = 4
n1 = 22, n2 = 47
n1 = 25, n2 = 15
n1 = 20, n2 = 21
n1 = 12, n2 = 27
n1 = 22, n2 = 22
n1 = 26, n2 = 14
n1 = 32, n2 = 48
n1 = 48, n2 = 36
Your thoughts are appreciated. Thank you so much!
FYI I have been using Excel's 2-sample t-test assuming unequal variance and Vassarstat's online Mann-Whitney test: http://vassarstats.net/index.html (under the heading Ordinal Data).
Best Answer
I don't think it's automatically reasonable to give a specific $n$ for this advice because there will be distributions/data sets that break it. Well, there might just be some sufficient $n$ but for me it doesn't kick in anywhere close to $n=40$.
In general, the answer is a very clear 'no'. The t tends to have reasonably good power relative to the MW for light-tailed distributions ... and can have really bad power for heavy-tailed ones. Skewness tends to be compounded with heavy tails - if power is your main motivation for using the t-test, you should probably avoid it in this case.
The Man-Whitney test should be sensitive to the sorts of departures you're trying to pick up (but note that scale and location will be confounded).
Another possibility is to do a permutation, randomization or bootstrap test.