Solved – What determines the precision of uncertainties

measurement errorprecisionuncertainty

What limits the precision with which you can describe the uncertainty of a measurement?

I will describe two examples that feel qualitatively different, but I am not sure if they are quantitatively different in how you would deal with the uncertainty.

Measuring the length of a pencil with a ruler that is graduated in units of cm. With such a measurement device you can accurately determine the length of the pencil down to the whole cm, and then you can estimate one additional decimal place further (mm). For example, one might determine that the pencil is 7.3 cm long. The uncertainty of the measurement lies with in tenths place value of the measurement. One estimate on the uncertainty might be .5 mm, since this is essentially the largest reasonable error one could make without causing the value to 'bleed over' into another mm unit. So one report the pencil's length as: 7.3 ± 0.05 cm. Here, only one measurement is being preformed on a single object. The uncertainty reported is the uncertainty in the last decimal place, and as such only has a single significant figure. It would not be reasonable to report values such as: 7.30 ± 0.05 cm or 7.3 ± 0.050 cm since these imply more precision than is possible to measure with this device.

Incidentally, in this example, what other ways might there by to estimate the uncertainty in the measurement besides that 'reasonable overestimate' of .5 mm that I described? Also, how does the uncertainty in the accuracy of the ruler come into play? In my example, I assumed that the ruler, essentially, gave us our definition of what a cm is, but in actuality the graduation markings on the ruler could be drawn with some error (albeit a small one).
Measuring the lengths of a set of nearly identical pencils. Through repeated measurements of these objects (one measurement per pencils, repeated for many pencil) one can collect a set of measurements for the length of this type of pencil. From this, one can determine the average length of this type of pencil (L). An estimate of the uncertainty in the length of this type of pencil is could be found as the standard deviation of the measured values (σ). At this point, one can report the pencils' lengths as: L ± σ. We will say that these measurements were preformed with the same ruler as in part 1, namely one graduated in units of cm, with individual measurements recorded with estimates down to mm precision. As such, L should have mm precision, at most (let's say 7.3 cm again). What about σ?

I see three possible scenarios:

A) The distribution of measurements is extremely tight, for example σ = 0.0000034873 cm (intentionally displaying something that mimics a calculator output). Here, the first significant digit of the uncertainty is for a value far smaller than the minimum precision of the measuring device. Would one report an uncertainty of 0 mm, or 3E-6 cm? Neither of these feel right. How does the uncertainty from (1) come into play?

B) The distribution of measurements is extremely broad, for example σ = 4.3289483 cm (obviously these are hardly nearly identical pencils anymore as was previously assumed). Now the most significant digit in the uncertainty is the same order of magnitude as the average. So one would then neglect the mm precision of the measurement and report the length to be 7 ± 4 cm? Here I round the average value to the same most significant decimal (whole cm, in this case).

C) The distribution is in between the two extreme cases mentioned above, for example σ = 0.295401 cm. Now it seems reasonable to simple round everything to the nearest mm and report the length as 7.3 ± 0.3 cm.

Would there ever be a reason where one is justified in reporting an uncertainty with more than one significant figure? For example, 7.3 ± 1.3 cm?

Best Answer

Your pencil example is peculiar. You'll see why when I describe how this precision thing works in a typical case.

Say, you're measuring a room with a measuring tape that has 1 mm ticks. You get 10033 mm measurement. The way to report this is 10033$\pm$0.5 mm. You usually take the half the tick as an a priori uncertainty $\sigma=0.5$.

To increase precision you measure the room several times: 10033, 10041, 10031. Now you can calculate the standard deviation $\sigma_3\approx 5.3$ mm, so you can throw out the a priori uncertainty, and report $10035\pm 5$. You see how $\sigma_3>\sigma$.

Your ruler is graded with 1 cm ticks then the a priori precision is usually reported as 0.5 cm, but you can certainly eye ball with close to 1 mm precision, so maybe 0.5 mm is an appropriate precision to report. Also, who would measure a pencil with 1 cm graded ruler? I would say it's not an appropriate instrument for the task. In fact I have never seen a ruler with 1 cm ticks, even the measuring tapes used in construction would have 1 mm marks.

Get a ruler with 1 mm grading or a standard caliper, where the a priori precision is going to be much lower than the uncertainty calculated from the few repeated measurement. This way you don't need to deal with interaction of your instrument's precision and the subject of your interest.

Related Solutions

Solved – Estimating error from repeated measurements

If you assume all the noises are Gaussian (and especially that +/- 0.1 really isn't... but anyway), then I think the 0.16 is already an estimate of the combination of the two noises. So I would report 4.32 +- 0.16/$\sqrt{n}$, where n is the number of measurements. A derivation:

So we're trying to measure the width $\mu$ of a block of wood. Suppose every measurement has some Gaussian error (perhaps because different people are measuring differently). We can write these as $X_{1},\ldots,X_{n}\sim\mathcal{N}(\mu,\sigma^{2})$. But the ruler itself has some imprecision, which for simplificity we'll model as Gaussian as well. So our final observations are $Y_{1},\ldots,Y_{n}$, distributed as $Y_{i}\sim\mathcal{N}(X_{i},\rho^{2})$. Suppose $\rho^{2}$ is known (e.g. the precision of the ruler). Then our goal is estimate $\mu$, and $\sigma^{2}$ is a nuisance parameter. This is easy because the $X$'s and $Y$'s are jointly normal. Write $(X,Y)$ for a generic pair of these variables. Their joint distribution is $$ \begin{pmatrix}X\\ Y \end{pmatrix}\sim\mathcal{N}\left(\begin{pmatrix}\mu\\ \mu \end{pmatrix},\begin{pmatrix}\sigma^{2} & \sigma^{2}\\ \sigma^{2} & \rho^{2}+\sigma^{2} \end{pmatrix}\right) $$ So $Y\sim N(\mu,\rho^{2}+\sigma^{2})$.

Let $\bar{Y}=(Y_{1}+\cdots+Y_{n})/n$ be the our estimator for $\mu$. Then $\bar{Y}\sim\mathcal{N}(\mu,[\rho^{2}+\sigma^{2}]/n)$. Since we don't know $\sigma^{2}$, I'd recommend plugging in your favorite sample estimate for the variance of $Y$, call it $\hat{\sigma}_{Y}^{2}$, and plugging that in for $\rho^{2}+\sigma^{2}$. So you end up with $\bar{Y}\pm\hat{\sigma}_{Y}/\sqrt{n}$.

Solved – How to compare measurements and uncertainties made with different measuring instruments

The model you use to "simulate your problem" can be used almost verbatim to estimate the parameters you are interested in using Bayesian estimation. Here is the model I'll use (using the same notation as you):

$$ L_B \sim \mathrm{Normal}(\mu, \sigma) \\ x_i \sim \mathrm{Normal}(\mu, \sigma) \mathrm{\ for\ i\ from\ 1\ to\ N} \\ L_{Ai} \sim \mathrm{Normal}(x_i \cdot \mathrm{gain} - \mathrm{offset}, \mathrm{dispersion}) \mathrm{\ for\ i\ from\ 1\ to\ N} \\ $$

The glaring omision in this model compared to your problem is that I don't include the assumption that some of the same $x_i$s that got measured by B could then be measured again by A. This could probably be added, but I'm not completely sure how.

This model is implemented in R & JAGS below using very vague, almost flat priors, the data used is the one you generated in your question:

library(rjags)

model_string <- "model{
  for(i in 1:length(L_B)) {
  L_B[i] ~ dnorm(mu, inv_sigma2) # <- reparameterizing sigma into precision 
                                 #    needed because of JAGS/BUGS legacy.  
  }
  for(i in 1:length(L_A)) {
    x[i] ~ dnorm(mu, inv_sigma2)
    L_A[i] ~ dnorm(gain * x[i] - offset , inv_dispersion2)
  }

  mu ~ dnorm(0, 0.00001)
  inv_sigma2 ~ dgamma(0.0001, 0.0001) 
  sigma <- sqrt(1 / inv_sigma2)
  gain ~ dnorm(0, 0.00001) T(0,)
  offset ~ dnorm(0, 0.00001)
  inv_dispersion2 ~ dgamma(0.0001, 0.0001)
  dispersion <- sqrt(1 / inv_dispersion2)
}"

Let's run it and see how well it does:

model <- jags.model(textConnection(model_string), list(L_A = L_A, L_B = L_B), n.chains=3)
update(model, 3000)
mcmc_samples <- coda.samples(model, c("mu", "sigma", "gain", "offset", "dispersion"), 200000, thin=100)
apply(as.matrix(mcmc_samples), 2, quantile, c(0.025, 0.5, 0.975))
##       dispersion   gain      mu   offset  sigma
## 2.5%     0.01057 0.1366 -0.3116 -0.51836 0.9365
## 50%      0.18657 1.0745 -0.1099 -0.26950 1.0675
## 97.5%    1.20153 1.2846  0.1051 -0.04409 1.2433

The resulting estimates are reasonably close to the values you used when you generated the data:

c(gain_A, offset_A, dispersion_A)
## [1]  1.1 -0.2  0.5

...except for, perhaps, dispersion. But with more data, perhaps more informed priors and running the MCMC sampling longer this estimate should be better.

Best Answer

Related Solutions

Solved – Estimating error from repeated measurements

Solved – How to compare measurements and uncertainties made with different measuring instruments

Related Question