Solved – two-sample t-test VS two one-sample t-tests. What’s the difference

r

I have dataset which contains values of some numeric variable for two independent groups:

mean sd n
group 1: 5.094294 0.9678155 17
group 2: 4.614275 0.9080703 230

So, I want to check hypothesis that the mean of group 1 is greater than the mean of group 2 with 95% confidence level.

  1. As far as I know, the most common way do this is two sample t-test. I'd checked it for both var.eual = TRUE and FALSE. This approach leads to accepting of my hypothesis (mean of gr1 is greater than mean of gr2).

t.test(gr1, gr2,paired = FALSE, var.equal = TRUE, alternative = "greater",conf.level = 0.95)

Two Sample t-test
data: gr1 and gr2
t = 2.0939, df = 245, p-value = 0.01865
alternative hypothesis: true difference in means is greater than 0
95 percent confidence interval:
0.1015146 Inf
sample estimates:
mean of x mean of y
5.094294 4.614275

t.test(gr1, gr2,paired = FALSE, var.equal = FALSE, alternative = "greater",conf.level = 0.95)

Welch Two Sample t-test
data: gr1 and gr2
t = 1.9815, df = 18.145, p-value = 0.03145
alternative hypothesis: true difference in means is greater than 0
95 percent confidence interval:
0.06013003 Inf
sample estimates:
mean of x mean of y
5.094294 4.614275

  1. On the other hand we can separately calculate 95% confidence interval for mean of each group. If this confidence intervals have intersection, then we'll be able to claim that means are not different (for 95% confidence level).
    This approach leads to rejection of my hypothesis.

t.test(gr1,alternative = "greater")

One Sample t-test
data: gr1
t = 21.7028, df = 16, p-value = 1.353e-13
alternative hypothesis: true mean is greater than 0
95 percent confidence interval:
4.684483 Inf
sample estimates:
mean of x
5.094294

t.test(gr2,alternative = "less")

One Sample t-test
data: gr2
t = 77.0634, df = 229, p-value = 1
alternative hypothesis: true mean is less than 0
95 percent confidence interval:
-Inf 4.713163
sample estimates:
mean of x
4.614275

Can somebody explain me why this two approaches give different results? Which of approaches is better to use and why?

Best Answer

The two-sample t-test is appropriate here, because you want to compare the two groups directly.

Two groups can differ significantly, and yet the CIs can still overlap. However, if the CIs do not overlap, then the groups must differ significantly. (This is of course assuming that the significance test and the CIs are calculated using the same assumptions about the data.) This is commonly misunderstood. Reference: http://blog.minitab.com/blog/real-world-quality-improvement/common-statistical-mistakes-you-should-avoid

How can the means of two groups differ significantly and yet have overlapping CIs? Loosely speaking, I think of it this way. There is 95% likelihood that the true mean for each group lies within the CI for that group. But in order for them to have the same mean, one group mean would lie at the extreme of its CI, and the other group mean would lie at the opposite extreme of its CI. That is an unlikely scenario.

Related Question