I did a Standard Error Plot of my two treatments, and homogeneity of variance appeared to be justified (there was only a slight difference in variance).
However, then I ran Levene's during my independent t-test and got a Sig. value of 0.000.
My sample size was in the thousands for each treatment, so I was wondering how does sample size affect the outcome of Levene's? My assumption is that: as sample size increases, the accuracy of the sample in relation to the population increases and therefore only a slight difference in variance would be really magnified.
Some clarification on this would be awesome, thanks!
Best Answer
Your intuition is correct. As with all tests, a higher $n$ will lead to higher power all other things being equal. A larger actual difference in variances obviously also makes it more likely that the Levene test can reject the null hypothesis. But with large $n$, even effects (here difference in variance) that are practically too small to worry about become statistically significant.
You will also note that using the Levene test this way (and I don't really know another way to use it) brings a fundamental conflict of interest: You want to prove the null hypothesis! You do this Levene test to conclude equality of variances, not to reject it. But you shouldn't try to prove the null hypothesis. Trying to prove the null hypothesis rewards lazy data collection or in your case penalizes abundant data.
If your primary goal is to learn about those variances, your best shot are equivalence testing or confidence interval based approaches. With a high $n$, you will have rejected $H_0 \quad \sigma_1=\sigma_2$, but you can also get an upper- and lower bound of that difference from the confidence interval. The larger your $n$, the more precisely you will know how much difference there is in variances.
It is more likely that you did that test as a precursor to another test that requires equality of variances. Know that unequal variances are only really a problem when the sample sizes are very different as well. Also, for t-tests for example you could just use the Welch version and not worry about variances at all.