Solved – Comparing the mean difference between data measured by different equipment, t-test suitable

equivalencemeanmeasurementt-test

My problem is as follows:

I have 2 measuring devices, both using different measuring techniques, each trying to quantify properties of objects passing their measurement area. The devices measure the objects at the same time. For this particular problem I'm only interested in the number of objects each device is able to measure.

For each time unit, which is set to one hour, the devices summarize the number of objects they've measured or "seen", like the below example data:

Time    Device1 Device2
00:00   58      47
01:00   38      52
02:00   12      13
03:00   0       2
04:00   23      2
....    ..      ..

I want to compare the mean difference of these 2 datasets

The first thing that came to my mind was to use a 2-tailed t-test to determine if there is a significant difference in means between the two datasets. Using the following parameters in Excel:

Hypothetized Mean Difference = 0
Alpha = 0,05

Question: Is the t-test even suitable in this particular setup? And if it is, is my data paired or independent? If not suitable, what other method could be used?

I guess what confuses me is the fact that I have 2 measuring devices measuring the same event, instead of one device measuring different events. I haven't dealt with statistical problems in ages, so my knowledge is a bit rusty. Any help would be greatly appreciated.

Edit: As has been pointed out in the answers, the question that I should be asking is: do the devices produce equivalent measurements? For clarification, the example data provided is fake, but closely resembles the measured data. The measured data is not normally distributed.

Best Answer

The issue is less with can one perform the statistical test (paired t test) for difference?, and more with what is the inference you are drawing from the tests results?

From what you describe, both measurements are made on the same objects at the same time, your test is trying to support inference for whether or not the equipment are making different measurements. A few thoughts:

  1. If you are interested in whether or not the two equipments make equivalent measurements—a separate statistical question than whether they make different measurements—you should also be performing equivalence tests, and combining your inferences from both tests.

  2. If you are interested in a deeper understanding of how the two measures perform, perhaps a regression model, such as OLS regression, which will tell you trend and strength of association, as well as evidence of association existing.

  3. From the tiny picture you give, your data do not really look normally distributed, which is one of the t test's assumptions. Perhaps a distribution-free test, such as the sign rank test would be appropriate?

Related Question