I am measuring a sample thickness using two different methods in potentially different locations. We are using the same equipment (G R&R came back OK), but we gather 360 data points using 1 technique, and 120 data points using the other technique. The datapoints are likely collected in different locations around the part.
The sampling process is non-destructive (and therefore does not modify the sample) and our samples do not change over time. One sampling method is significantly quicker than the other. I would like to compare means, knowing that they will be slightly different, but we are okay if we are off slightly.
We have 73 different samples, each independent from one another. Would I use a paired t-test or one-way anova here (or would it be repeated measure ANOVA)?
I compared the data using a Pair-t test and it comes back with a P-value of 0.018 (reject the null which leads to the conclusion that the means are statistically different, but they are not significantly different).
Paired T-Test and CI: C2, C8
Paired T for C2 – C8
N Mean StDev SE Mean
C2 73 2809.1 126.6 14.8
C8 73 2802.5 126.3 14.8
Difference 73 6.63 23.38 2.74
99% CI for mean difference: (-0.61, 13.87)
T-Test of mean difference = 0 (vs ≠ 0): T-Value = 2.42 P-Value = 0.018
A one-way anova indicates that we fail to reject the null. However, I am under the impression that the samples can not be considered independent, because they are the same sample, but we could be measuring them in different locations. Would ANOVA be the wrong technique to use here? Or because we are measuring different locations on the same part, we should assume that the measurements are independent and therefore a paired-t test should NOT be used here.
http://www.statstutor.ac.uk/resources/uploaded/paired-t-test.pdf
Makes me think we should be using a paired t-test.
• A comparison of two different methods of measurement or two different treatments
where the measurements/treatments are applied to the same subjects (e.g. blood
pressure measurements using a stethoscope and a dynamap).
Best Answer
As the population of samples that are being collected are from the same object, you need to consider a paired t-test with dependent samples here. First and foremost, you need to check if normality assumptions hold true. One possible way to do that is to generate Q-Q plots and see how the data is distributed.
If the normality assumption does not hold true, then consider transforming the data (either $\log$ or $\exp$). Check again for normality using Q-Q plots. If normality fails, then look at non-parametric approaches such as Wilcoxon tests. They do not assume things about the distribution of the data.
A simple way to study these methods is by using G*Power 3.1 toolbox. They have a very simple user-interface that will allow you to select the type of test and generate a $p$-value.