Solved – paired t-test for multiple groups

anovapaired-comparisonst-test

I have had a particular problem that I haven't been able to solve.

Here is the scenario: I would like to perform a paired t-test of sorts on multiple groups. Instead of before and after conditions, there are multiple conditions (ex: treatment 1, treatment 2, treatment 3 etc). Each subject appears in each 'group'. I have tried to use ANOVA, but this method averages data of all subjects within each condition, then compares their means. However, I would like to look at the differences of individual subjects between the conditions, and compare those between subjects. To me, it sounds like a paired t-test for of 2+ groups. I am using n=4. I hope that makes sense.

thanks for any input

Best Answer

The test you are looking for is one-way repeated measures ANOVA. That is equivalent to two-way ANOVA, with repeated measures in one factor, with no replication. Following the ANOVA, you'll want to run followup tests to make the comparisons you are asking for, while adjusting for multiple comparisons.

Related Solutions

Solved – When does an unpaired test result in higher p-value than a paired test

As @whuber says in the comment above, when the measures are negatively correlated, the p-value can be lower in the unpaired test than in the paired test. Here's an example where there is a difference:
```
library(MASS)
s  <- matrix(c(1, -0.8, -0.8, 1), 2)
df <- mvrnorm(n=100, mu=c(0, 0.3), Sigma=s, empirical=TRUE)
t.test(df[,1], df[, 2], paired=FALSE)
t.test(df[,1], df[, 2], paired=TRUE)
```
The first test (unpaired) gives p=0.035, the second gives p=0.117.
Yes, this is a design issue. This book chapter discusses it: Keren, G. (2014). Between-or within-subjects design: A methodological dilemma. A Handbook for Data Analysis in the Behaviorial Sciences: Volume 1: Methodological Issues Volume 2: Statistical Issues, 257, which you can read some of on Google books.

Hmmm... I'm not sure. I'd do a simulation to find out the effect on the type I error rate. How this affects your power is a separate issue that I haven't looked into here. Slight adaptation of my previous code:

paired   <- rep(NA, 1000)
unpaired <- rep(NA, 1000)
for(i in 1:1000){
      df          <- mvrnorm(n=100, mu=c(0, 0), Sigma=s, empirical=FALSE)
      unpaired[i] <- t.test(df[,1], df[, 2], paired=FALSE)$p.value
      paired[i]   <- t.test(df[,1], df[, 2], paired=TRUE )$p.value
}

sum(paired < 0.05)
sum(unpaired < 0.05)

Result:

> sum(paired < 0.05)
[1] 46
> sum(unpaired < 0.05)
[1] 137

Well look at that. If you treat them as unpaired, your type I error rate rockets. You need to treat them as paired to get the right answer. I believe (it's a long time since I've read it) that this is one of the issues Keren talks about in that chapter. If you're going to have data that might be negative correlated (e.g. amount of soup and amount of burgers someone eats) you'll have more power with an unpaired design.

Solved – Matched data: Paired t-test vs. indpendent

A two-sample $t$-test seems more reasonable here, predominantly because of the following reason.

If you were to do this experiment again, and match group(X) and control(Y), it is unlikely that the exact same participants will be matched. In fact, if you have 2 people in group(X) who have exactly the same demographics, you can match both these participants with the same control. Thus, matching here will not be helpful.

If you want to study the effect of the treatment as a function of age, sex, etc, you should consider regression with these demographics as covariates and observed blood pressure as response.

When doing a two samples $t$-test, you are assuming the two samples are not dependent on each other. This assumption will not be violated just because the participants had a similar demographics. If you went out and obtained the $Y$ randomly and the $X$ randomly, and assigned them treatment randomly, then there should be no problem with the independence assumption.

Best Answer

Related Solutions

Solved – When does an unpaired test result in higher p-value than a paired test

Solved – Matched data: Paired t-test vs. indpendent

Related Question