Solved – Comparison among 3 paired data sets. (paired t-test, ANOVA?)

anovaMATLABpaired-comparisonspaired-datasas

I measured 30 samples each with 3 devices. The devices are the same make and model, just different units. How do I statistically test if the devices measurements are the same or not? If I had two data sets to compare, a simple paired t-test works great. I've read that running three paired t-tests (1vs2, 2vs3, and 3vs1) separately to draw conclusions is incorrect…the familywise error is higher than the error of the individual comparisons. But when I set-up ANOVA it doesn't pair the data by sample, it just lumps everything into a single average, and the effect of each device is lost.

So what test do I use?

Best Answer

You seem to be conflating the terms paired and pairwise. t-tests are pairwise when you have multiple groups that you compare to each other. They are paired if you have in each group the same number of measurements with matched sample points (within subjects). In your case, it is paired and pairwise, but the concepts are different.

There are within subject Anovas that consider the data paired. I'm not an expert here, since I can't use them in my experiments. In general, you should know that Anova + post hoc t-tests are often bundled for convenience, not for binding statistical reasons. The Anova itself cannot tell you very much. Only if there are significant differences among some of the groups or not. If you want information about specific pairs of groups, look at the t-tests.

Don't compute the differences before handing the data to an ANOVA or you will be testing for equality of differences where you want to test for equality of measures. (Unless the SAS documentation explicitly says so, which I don't believe.)

For the post-hoc tests, just subtract the two columns before entering the difference as one column into the t-test. In addition, you're doing pairwise t-tests which means that you also need to control the family wise error rate with a Bonferroni correction or Holm's method. Those two things are independent from each other and the t-tests are also independent from the ANOVA. So if you want to to the Anova and t-tests, better not use the Tuckeys tests, you will lose power. Even if they come with the Anova (which they shouldn't with a within subjects Anova), you can simply throw them out and do your own.

Related Question