I have run Levene's and Bartlett's test on groups of data from one of my experiments to validate that I am not violating ANOVA's assumption of homogeneity of variances. I'd like to check with you guys that I'm not making any wrong assumptions, if you don't mind 😀
The p-value returned by both of those tests is the probability that my data, if it were generated again using equal variances, would be the same. Thus, using those tests, to be able to say that I do not violate ANOVA's assumption of homogeneity of variances, I would only need a p-value that is higher than a chosen alpha level (say 0.05)?
E.g., with the data I am currently using, the Bartlett's test returns p=0.57, while the Levene's test (well they call it a Brown-Forsythe Levene-type test) gives a p=0.95. That means, no matter which test I use, I can say that the data I meet the assumption. Am I making any mistake?
Thanks.
Best Answer
The p-value of your significance test can be interpreted as the probability of observing the value of the relevant statistic as or more extreme than the value you actually observed, given that the null hypothesis is true. (note that the p-value makes no reference to what values of the statistic are likely under the alternative hypothesis)
EDIT: in mathematical terminology, this can be written as: $$p-value = Pr(T > T_{obs} | H_{0})$$ where $T$ is some function of the data (the "statistic") and $T_{obs}$ is the actual value of $T$ observed; $H_{0}$ denotes the conditions implied by the null hypothesis on the sampling distribution of $T$.
You can never be sure that you're assumptions hold true, only whether or not the data you observed is consistent with your assumptions. A p-value gives a rough measure of this consistency.
A p-value does not give the probability that the same data will be observed, only the probability that the value of the statistic is as or more extreme to the value observed, given the null hypothesis.