I'm trying to understand the meaning of stating the standard deviation (SD) of the mean difference (MD) [or otherwise called the absolute mean difference]. This is for a paper I'm writing up where other examples also quote the SD of the MD as part of the analysis summary.
While I understand what the mean difference represents, i.e., being a "measure of statistical dispersion", I'm not sure of the utility of reporting the standard deviation of this value. I can kind of conceptualise what it means, but again the issue is the utility.
For reference, the analysis is of repeated measures data for equivalence. I will be reporting the correlation coefficient and also the statistical probability for equivalence (with equivalence range) using a Two One Sided T-test (TOST). Thus my question regarding the utility of the standard deviation of the mean difference.
Maybe I'm lacking in statistical knowledge and please feel free to point out a suitable reference for me to consult.
Thanks in advance.
Update – example:
Machine A is the reference measurement system. Machine B is the new measurement system. A certain number of real life measurements are made with each machine (N) on a common object to understand how equivalent the machines are. Ideally they're identical – "of course" say the designers, and can even be expected – but this testing will be used to support the equivalence limit of +/- x units. (Given in reality it's not possible to say A is exactly equivalent to B.) So a paired measurement comparison.
When reporting these results it has been common to report the mean difference and the standard deviation of the mean difference, plus the correlation coefficient (Pearson's) with its p value.
Update – comments:
1. This question does indeed refer to the standard deviation of the mean difference, not simply the difference.
2. I well understand what the standard deviation means in terms of distributions. But my question is really "what is the meaning of the standard deviation of the mean difference in relation to the quantity being measured?"
3. If it's as simple as a guide to the variability of the difference, then so be it.
4. If the result was reported as A = B +/- x units with a p < 0.05 then I understand that this is different to quoting a standard deviation type number, but I still get it intuitively. I don't get the SD of a MD in regard to the original measurement entity nearly as well.
Best Answer
Think of the difference like any other statistic that you are collecting. These differences are just some values that you have recorded. You calculate their mean and standard deviation to understand how they are spread (for example, in relation to 0) in a unit-independent fashion.
The usefulness of the SD is in its popularity -- if you tell me your mean and SD, I have a better understanding of the data than if you tell me the results of a TOST that I would have to look up first.
Also, I'm not sure how the difference and its SD relate to a correlation coefficient (I assume that you refer to the correlation between two variables for which you also calculate the pairwise differences). These are two very different things. You can have no correlation but a significant MD, or vice versa, or both, or none.
By the way, do you mean the standard deviation of the mean difference or standard deviation of the difference?
Update
OK, so what is the difference between SD of the difference and SD of the mean?
The former tells you something about how the measurements are spread; it is an estimator of the SD in the population. That is, when you do a single measurement in A and in B, how much will the difference A-B vary around its mean?
The latter tells us something about how well you were able to estimate the mean difference between the machines. This is why "standard difference of the mean" is sometimes referred to as "standard error of the mean". It depends on how many measurements you have performed: Since you divide by $\sqrt{n}$, the more measurements you have, the smaller the value of the SD of the mean difference will be.
SD of the difference will answer the question "how much does the discrepancy between A and B vary (in reality) between measurements"?
SD of the mean difference will answer the question "how confident are you about the mean difference you have measured"? (Then again, I think confidence intervals would be more appropriate.)
So depending on the context of your work, the latter might be more relevant for the reader. "Oh" - so the reviewer thinks - "they found that the difference between A and B is x. Are they sure about that? What is the SD of the mean difference?"
There is also a second reason to include this value. You see, if reporting a certain statistic in a certain field is common, it is a dumb thing to not report it, because not reporting it raises questions in the reviewer's mind whether you are not hiding something. But you are free to comment on the usefulness of this value.