Solved – Two test groups with multiple measurements vs a single reference value

anovahypothesis testingmultiple-comparisonst-test

This is my scenario:

I'm testing two length measuring devices. I'm measuring a model that has notches at different lengths in order to collect 15 different measurements.
I know the "real" value for each distance in order to calculate 15 "errors" for each device.
The same 15 measurements are repeated ten times for each device.

  • Device A – 15 measurements x 10 times
  • Device B – 15 measurements x 10 times

In order to have a general idea about which one is better I thought that a t-test would be ok (tell me if not): I put all the errors of Device A together and compare them with B.

If I want to compare A vs B of each one of the 15 measurements would it be ok to do a one way ANOVA? I'm asking it because I have only two groups.

Best Answer

If you just want to compare the differences between the two groups than a hypothesis test like a t-test or a Wilcoxon test is the most convenient way. There are some differences between statistical tests regarding small sample properties and how they deal with different variances. For reasons of simplicity I propose a simple t-test (welche two sample t-test).

Let´s have a look a two vectors. The first vector is called "a".

2 4 3 5 6 4 2 7 8 4

The second vector is called "b".

6 3 4 2 6 8 8 6 8 4

So you can use the following R command for testing. 

t.test(a,b)

    Welch Two Sample t-test

data:  a and b
t = -1.0674, df = 17.897, p-value = 0.3
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -2.9691637  0.9691637
sample estimates:
mean of x mean of y 
      4.5       5.5

The null hypothesis is that both samples have the same mean. The alternative hypothesis is that there are significant differences between the values of the two vectors.

One-way ANOVA however is applicable if you want to compare means of three or more samples. As you have only two samples you should not use a one-way ANOVA.

Related Question