While I am doing post hoc test for one way ANOVA, I am getting significant results for LSD but the same variables does not have significant values for Bonferroni. Why is it so?
Solved – post hoc tests in ANOVA
anovabonferroni
Related Solutions
It depends on what you plan to do with the result.
If you want to form a conclusion regarding a null hypothesis and have a specifiable rate of false positives (i.e. you want a Neyman-Pearson hypothesis test) then no, you can't do it. Spuriously 'significant' results turn up all the time and any result that looks significant will quite likely turn out to be statistically significant even when it is not real. Neyman-Pearson analysis allows you to test PREDEFINED hypotheses with PREDETERMINED analyses.
If you want to use your data to help form hypotheses to test with new experiments (i.e. use a Fisherian approach) then yes, go ahead and test! The fact that you cannot reliably test an hypothesis that is formed on the basis of seeing a dataset using that same dataset does not mean that the dataset cannot point to something worthy of a follow-up experiment.
Indeed an omnibus test is not strictly needed in that particular scenario and multiple inference procedures like Bonferroni or Bonferroni-Holm are not limited to an ANOVA/mean comparison settings. They are often presented as post-hoc tests in textbooks or associated with ANOVA in statistical software but if you look up papers on the topic (e.g. Holm, 1979), you will find out that they were originally discussed in a much broader context and you certainly can “skip the ANOVA” if you wish.
One reason people still run ANOVAs is that pairwise comparisons with something like a Bonferroni adjustment have lower power (sometimes much lower). Tukey HSD and the omnibus test can have higher power and even if the pairwise comparisons do not reveal anything, the ANOVA F-test is already a result. If you work with small and haphazardly defined samples and are just looking for some publishable p-value, as many people are, this makes it attractive even if you always intended to do pairwise comparisons as well.
Also, if you really care about any possible difference (as opposed to specific pairwise comparisons or knowing which means differ), then the ANOVA omnibus test is really the test you want. Similarly, multi-way ANOVA procedures conveniently provide tests of main effects and interactions that can be more directly interesting than a bunch of pairwise comparisons (planned contrasts can address the same kind of questions but are more complicated to set up). In psychology for example, omnibus tests are often thought of as the main results of an experiment, with multiple comparisons only regarded as adjuncts.
Finally, many people are happy with this routine (ANOVA followed by post-hoc tests) and simply don't know that the Bonferroni inequalities are very general results that have nothing to do with ANOVA, that you can also run more focused planned comparisons or do a whole lot of things beside performing tests. It's certainly not easy to realize this if you are working from some of the most popular “cookbooks” in applied disciplines and that explains many common practices (even if it does not quite justify them).
Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6 (2), 65–70.
Best Answer
Bonferroni adjusts For Type I error rates. LSD does not. With more post hoc comparisons, more are likely to be significant using LSD. If making more than 3 pairwise comparisons, don't use LSD. I recommend HSD (Tukey) in SPSS if homogeneity of variances is not violated. Bonferroni is very conservative.