Like other parametric tests, the analysis of variance assumes that the data fit the normal distribution. If your measurement variable is not normally distributed, you may be increasing your chance of a false positive result if you analyze the data with an anova or other test that assumes normality. Fortunately, an anova is not very sensitive to moderate deviations from normality; simulation studies, using a variety of non-normal distributions, have shown that the false positive rate is not affected very much by this violation of the assumption (Glass et al. 1972, Harwell et al. 1992, Lix et al. 1996). This is because when you take a large number of random samples from a population, the means of those samples are approximately normally distributed even when the population is not normal.
It is possible to test the goodness-of-fit of a data set to the normal distribution. I do not suggest that you do this, because many data sets that are significantly non-normal would be perfectly appropriate for an anova.
Instead, if you have a large enough data set, I suggest you just look at the frequency histogram. If it looks more-or-less normal, go ahead and perform an anova. If it looks like a normal distribution that has been pushed to one side, like the sulphate data above, you should try different data transformations and see if any of them make the histogram look more normal. If that doesn't work, and the data still look severely non-normal, it's probably still okay to analyze the data using an anova. However, you may want to analyze it using a non-parametric test. Just about every parametric statistical test has a non-parametric substitute, such as the Kruskal–Wallis test instead of a one-way anova, Wilcoxon signed-rank test instead of a paired t-test, and Spearman rank correlation instead of linear regression. These non-parametric tests do not assume that the data fit the normal distribution. They do assume that the data in different groups have the same distribution as each other, however; if different groups have different shaped distributions (for example, one is skewed to the left, another is skewed to the right), a non-parametric test may not be any better than a parametric one.
References
- Glass, G.V., P.D. Peckham, and J.R. Sanders. 1972. Consequences of failure to meet assumptions underlying fixed effects analyses of variance and covariance. Rev. Educ. Res. 42: 237-288.
- Harwell, M.R., E.N. Rubinstein, W.S. Hayes, and C.C. Olds. 1992. Summarizing Monte Carlo results in methodological research: the one- and two-factor fixed effects ANOVA cases. J. Educ. Stat. 17: 315-339.
- Lix, L.M., J.C. Keselman, and H.J. Keselman. 1996. Consequences of assumption violations revisited: A quantitative review of alternatives to the one-way analysis of variance F test. Rev. Educ. Res. 66: 579-619.
In order to make sure that I can use parametric test, I need to make sure that my residual distribution is normal.
There is really no way to demonstrate that you have exact normality, but that's okay because approximate normality will generally be sufficient for hypothesis tests in regression to work the way you want.
However, when I refer to the value of skewness and kurtosis of the residual, it is -0.017 and -0.438 respectively, where i think this is considered as normal.
You can obtain values like that with residuals from a simple regression on normal data, but the kurtosis is just significant at the 5% level.
(Technical aside: I used simulation to assess the significance of the kurtosis of residuals here; not knowing the number of predictors, I did it for both independent normals and for one predictor at the given sample size, both showed essentially the same p-value; results should be similar for regression with small numbers of predictors.)
This doesn't actually suggest a problem with the inference when doing a regression or correlation, however. Your data won't be exactly normal; the essential question is 'are the data so badly non-normal that the inference no longer has the properties you wish?'
Unfortunately, when i do kolmogorov-smirnov, the significant value is 0.021, which indicates the residual is not normal.
What were the specified population mean and variance of the residuals for your KS test and how did you get such population values?
Could anybody please explain to me what to do.
I suggest you don't do a hypothesis test to assess the suitability of the assumption of normality, but instead to look at diagnostic displays that show you how badly non-normal the data are.
Some pointers -
See the points here
Also see the discussion on this question
See the comments under this answer,
and the advice in this answer
Consider this advice
Best Answer
Don't look at it as a binary thing: "either I can trust the results or I can't." Look at it as a spectrum. With all assumptions perfectly satisfied (including the in most cases crucial one of random sampling), statistics such as F- and p-values will allow you to make accurate sample-to-population inferences. The farther one gets from that situation, the more skeptical one should be about such results. You've got a substantial degree of nonnormality; that's one strike against accuracy. Now how about the other assumptions underlying the use of ANOVA? Size it all up the best you can, and document in a footnote or a technical section what you find. You also should look at this page, as @William pointed out.
As to your last question, I don't believe you need to change your strategy vis-a-vis multiple comparisons just because you move from a parametric to a nonparametric test. If you want to describe the rationale for your current approach, I'm sure people will be glad to comment on it.