[Physics] Multiple measurements of the same quantity – combining uncertainties

error analysisexperimental-physicsexperimental-technique

I have a number of measurements of the same quantity (in this case, the speed of sound in a material). Each of these measurements has their own uncertainty.

$$ v_{1} \pm \Delta v_{1} $$
$$ v_{2} \pm \Delta v_{2} $$
$$ v_{3} \pm \Delta v_{3} $$
$$ \vdots $$
$$ v_{N} \pm \Delta v_{N} $$

Since they're measurements of the same quantity, all the values of $v$ are roughly equal. I can, of course, calculate the mean:

$$ v = \frac{\sum_{i=1}^N v_{i}}{N}$$

What would the uncertainty in $v$ be? In the limit that all the $\Delta v_i$ are small, then $\Delta v$ should be the standard deviation of the $v_i$. If the $\Delta v_i$ are large, then $\Delta v$ should be something like $\sqrt{\frac{\sum_i \Delta v_i^2}{N}}$, right?

So what is the formula for combining these uncertainties? I don't think it's the one given in this answer (though I may be wrong) because it doesn't look like it behaves like I'd expect in the above limits (specifically, if the $\Delta v_i$ are zero then that formula gives $\Delta v = 0$, not the standard deviation of the $v_i$).

Best Answer

When you're combining measurements with different uncertainties, taking the mean is not the right thing to do. (Well, it's good enough if the uncertainties are almost the same.)

The right thing to do is chi-squared analysis, which gives a higher weight to the more accurate measurements. Here's how you do it:

$$\chi^2 = \sum \frac{(\text{observed value} - \text{true value})^2}{\text{(uncertainty associated with that observation)}^2}$$

You numerically choose the "true value" that minimizes $\chi^2$. That's your best guess.

Next, use the chi-square distribution to calculate the p-value (assuming the best guess is right). (Degrees of freedom is one less than the number of observations.) This will tell you whether your uncertainties were reasonable or whether you underestimated them. For example, if one measurement is $5.0 \pm 0.1$, and another measurement is $10.0 \pm 0.1$, then you probably underestimated your uncertainties.

IF you underestimated your uncertainties -- which is not unusual in practice -- then the right thing to do is to figure out where you went wrong in your uncertainty estimation, and correct the mistake. But there is a lazier alternative too, which is often good enough if the stakes are low: You can scale up all the uncertainties by the same factor until you get a reasonable $\chi^2$ p-value, say 0.5.

OK, now you have plausible measurement uncertainties, either because you did from the beginning or because you scaled them up. Next, you try varying the "true value" until the p-value dips down below, say, 5%. This procedure gives you lower bound and upper bound error bars on your final best-guess measurement.

I haven't done this in many years, sorry for any mis-remembering. I think it's discussed in Bevington&Robinson.

Related Question