T-Test Comparison – Difference Between Paired T-Test and Two-Sample T-Test

t-test

I have a simple question regarding the use of paired t-test and two-sample t-test:

  1. What is the consequence of using the independent two-sample t-test for dependent paired samples?
  2. What is the consequence of using the paired t-test for independent samples?

Thanks!

Best Answer

It would be bad -- very bad. If you ignore the pairing and use a two-sample t-test where you should have used the paired, chances are that you will not be able to detect the effect of interest. In this case, between subject variation is included in the estimate of variance used to measure the effect of interest. The variance is inflated, and only a substantial effect size will be deemed significant. Example: students are given a math test, followed by an instruction module, and then another test. If you ignore the pairing (before and after for each student), difference in ability is added to the effect of the module, thus making the effect harder to detect.

On the other hand, if you foist pairing onto an unpaired sample you reduce the degrees of freedom of the test. You are adding variables to the model (the pairing effect) that are basically random. In a sense, for each pair, you are adding a parameter for that pair -- but these parameters mean nothing. You would be adding noise, in a sense.

Related Question