I'm trying to run a t-test using t.test() but am confused about the p-value and confidence interval. Looking at the output I see a p-value of ~0.03. That leads me to believe I'd reject the null hypothesis at a 95% level. However, when I compare the difference in my means, 0.25, is solidly within the 95% confidence interval shown. With a small p-value I thought it would be outside, but I'm new to this whole process. How do I reconcile a small p-value with a difference within the confidence interval? Is the difference in mean not what the confidence interval shows?
> t.test(childs ~ vet, data = cols)
data: childs by vet
t = 2.163, df = 130.97, p-value = 0.03236
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
0.02163258 0.48494263
sample estimates:
mean in group FALSE mean in group TRUE
2.707134 2.453846
> 2.707 - 2.4538
[1] 0.2532
Best Answer
.2532 is always going to be in the confidence interval since this is how the interval was constructed. The formula is
$(\bar{x_1} - \bar{x_2}) \pm t^*_{df} \times SE(\bar{x_1} - \bar{x_2})$
Further more $\bar{x_1} - \bar{x_2}$ is called your observed result (or sample difference in means) and as you can see this is where the interval will be centered and thus always be contained in the confidence interval.
The hypotheses you are testing are
$H_0: \mu_1 - \mu_2 = 0$ vs. $H_1: \mu_1 - \mu_2 \neq 0$
Therefore, you when checking whether or not the null hypothesis is rejected based on the confidence interval you look to see if the null hypothesized value is in the confidence interval. In this case it is 0, since 0 is not in the interval we reject the null hypothesis at the .05 significance level. This is consistent with the decision made from the p-value.
Hopefully this helps but my response is very much an overview and there is a lot more going on conceptually here that I did not go into.