T-Test – Explaining Why p-Value is Significant but Confidence Interval Includes Zero

confidence intervalp-valuert-test

I am interested in learning whether the two groups below are statistically significant:

ex0112 = (
   BP       Diet
1   8    FishOil
2  13    FishOil
3  10    FishOil
4  14    FishOil
5   2    FishOil
6   1    FishOil
7   0    FishOil
8  -6 RegularOil
9   1 RegularOil
10  1 RegularOil
11  2 RegularOil
12 -3 RegularOil
13 -4 RegularOil
14  3 RegularOil

I run the following t-test:

t.test(BP ~ Diet, data = ex0112, conf.level = 0.99)

The results are:

    Welch Two Sample t-test

data:  BP by Diet
t = 3.0109, df = 9.7071, p-value = 0.01352
alternative hypothesis: true difference in means between group FishOil and group RegularOil is not equal to 0
99 percent confidence interval:
 -0.4610773 15.8896488
sample estimates:
   mean in group FishOil mean in group RegularOil 
               6.8571429               -0.8571429

As you can see, the p-value for this test is

p-value = 0.014

which is significant at .05, but the confidence interval is

99 percent confidence interval:
 -0.4610773 15.8896488

which includes zero, and hence makes the difference between the groups insignificant. How can this be explained?

Best Answer

You're calculating a $99\,\%$-confidence interval. If you decided on a significance level of $0.01$, you would not reject the null hypothesis based on the $p$-value, because it's larger than $0.01$. If you want to use a significance level of $0.05$, corresponding to a $95\,\%$-confidence interval, specify conf.level = 0.95:

dat <- data.frame(
  BP = c(8, 13, 10, 14, 2, 1, 0, -6, 1, 1, 2, -3, -4, 3)
  , Diet = rep(c("FishOil", "RegularOil"), each = 7)
)

t.test(BP~Diet, data = dat, conf.level = 0.95)

data:  BP by Diet
t = 3.0109, df = 9.7071, p-value = 0.01352
alternative hypothesis: true difference in means between group FishOil and group RegularOil is not equal to 0
95 percent confidence interval:
  1.98202 13.44655
sample estimates:
   mean in group FishOil mean in group RegularOil 
               6.8571429               -0.8571429

The $95\,\%$-confidence interval $(1.98; 13.45)$ does not include $0$, as expected.

Related Question