I gather data from two different groups of students: a experimental group and a control group. The experimental group received an intervention while the other one none. I am trying to figure out what can be interpreted from those results. I heard contradictory interpretation basically one stating that the null hypothesis can be rejected, the next one the null hypothesis cannot be rejected. Here are the results:
My interpretation:
(1) we can observe a numerical difference equal to about 2 points and a SD difference about 1 point.
(2) I understood that the t-test assume that SD is about the same.
LEVINE TEST to check equality of variance or Standard Deviation (SD)
(3) Levine test of equality variance (Ho: variance are equal, H1: variance are not equal) show a F-value F=1.434.
(4) As the ratio F (variance of the means: average of the sample variances) is greater than 1 => numerator grater than denominator => variances cannot be assumed to be equal.
(5) However, Given that p-value sig=0.240 > 0.05 , It is therefore not statistically different.
T-Test to check whether the means are statistically different
(7) T test: Ho: means are equal, H1: means are not equal.
(8) Because sig is greater than 0.05 the null hypothesis cannot be rejected.
(6) Therefore, we cannot interpret the difference in mean as statistically significant.
(7) It follows that we cannot ascertain with 95% certainty that the intervention on the students had a significant effect.
I have two questions:
(A)I am stuck with my statement for levene test (4) and (5). Many argue that you just have to look at the p-value of sig=0.240 > 0.05, therefore variance can be assumed to be equal. But the F value shows that the variance are different….?
(B) What can be interpreted from the 95% confidence interval of the difference? the lower being -4.861 and upper 1.241 ? Can I say that that there is a greater difference in means with the lower scores than the higher scores?
very long time I dealt with Stats. Very grateful for any help or insight.
Best Answer
A) Levene's test tests whether the variances differ between the two groups in the population. Of course, the variances in your sample will be different (i.e., leading to F-statistics greater than 1); Levene's test tests whether the difference in your sample is so extreme as to indicate a difference in the population. Here, because the p-value resulting from Levene's test is greater than .05, there isn't enough evidence to say that the variances differ in the population. Note that the assumptions of independent samples t-test that the variances must be equal refers to the population variances, not the sample variances. That said, all this is somewhat inconsequential because you should always proceed not assuming equal variance (though this is not a universally agreed upon statement), and, in your case, the result of your test for group differences in the mean is the same whether you make that assumption or not.
B) Interpreting confidence intervals, in general, is tricky. In a very basic way, it's a broader estimate of where the population mean difference is (i.e., broader than a point estimate). So an intuitive (though not completely correct) interpretation is that the population mean difference is probably somewhere within the interval. Precisely when your p-value is less than .05, your confidence interval will exclude 0. For testing the null hypothesis of a population mean difference of 0, it provides the same information as the p-value.
A more technical interpretation of confidence intervals more broadly (i.e., not the specific values you found) is that, if you were to repeatedly draw random samples from your population of interest and compute confidence intervals using the same procedure used in this analysis, the collection of confidence intervals would contain the population mean difference 95% of the time. That doesn't tell you much about your specific confidence interval, except that if it's one of those 95%, the population mean difference is somewhere within it. Don't try to interpret the confidence interval if you're not comfortable with statistics; it tells you way less about your question of interest than you might think.