I have an experiment where I'm testing the effect of a drug on behavior, measured in seconds. My drug group doesn't have normal distribution, probably because of a bimodal pattern: 6 of the 8 are clustered (between 100 and 150 s), while the other 2 are much higher (275 and 300 s). We believe this is because the drug worked for those 6, and not the others.
Furthermore, a One-Way Anova between the 3 groups failed Levene's test for equal variances (I also ran Welch's t-test, which was significantly diff.) However, an independent T-test between the 2 groups that I'm looking at shows no sig. diff. in Levene's Test, and the One-way Anova says there is a significant decrease in time between this group and a control group using Tukey's, which is what I'm looking for (the 3rd group is another control, but I'm not really interested in that one).
All 3 have an n= 8.
Where do I go from here?
Edit:
My design involves 24 animals split into 3 drug groups, 8 each: a control drug, saline, and the drug I am testing. I am seeing if there's a significant decrease in time between the drug group I'm testing and the saline group.
Best Answer
If the assumption of normality for one-way ANOVA does not hold, you can turn to a nonparametric analog to the one-way ANOVA: the Kruskal-Wallis test. Just as the assumption of normality underlying the unpaired t test may not be met, thus motivating the use of the rank sum test, onne can then use Dunn's test, or the more powerful (but less well known) Conover-Iman test to conduct post hoc pairwise tests if one rejects the omnibus Kruskal-Wallis test's null hypothesis.
In their most general form the nonparametric tests (Kruskal Wallis, rank sum, Dunn's, etc.) do not assume equal variances among groups. Instead, they test:
$$H_{0}:P(X_{A}>X_{B})=0.5$$
with
$$H_{a}:P(X_{A}>X_{B})\ne0.5$$
Or in words: the null hypothesis is that the probability that a randomly selected observation from group A is greater than a randomly selected observation from group B equals one half. The alternative is that the probability is not one half. For the Kruskal-Wallis test, the null hypothesis is that the probability that a randomly selected value from any group is greater than a randomly selected observation from any other group equals one half, with the alternative that at least one group that has a probability not equal to one half for being greater than a randomly selected value from another group.
One can interpret these as tests of location shift, median difference, or mean difference if the variances for all groups are all equal and the shapes of the distribution are the same (this is a pretty stringent requirement!), but nonparametric tests do not require such assumptions to use.
I have published a software package to perform Dunn's test for R (dunn.test), and Dunn's test for Stata (dunntest), and a software package to perform the Conover-Iman test for R (conover.test), and the Conover-Iman test for Stata (conovertest). Both packages correct for ties, and implement an array of familywise error rate and false discovery rate adjustments for multiple comparisons.
References
Dunn, O. J. (1964). Multiple comparisons using rank sums. Technometrics, 6(3):241–252.
Conover, W. J. (1999). Practical Nonparametric Statistics. Wiley, Hoboken, NJ, 3rd edition.
Conover, W. J. and Iman, R. L. (1979). On multiple-comparisons procedures. Technical Report LA-7677-MS, Los Alamos Scientific Laboratory.