What is an estimator of standard deviation of standard deviation if normality of data can be assumed?
Solved – Standard deviation of standard deviation
estimationnormality-assumptionstandard deviation
Related Solutions
If all your measurements are using the same units, then you've already addressed the scale problem; what's bugging you is degrees of freedom and precision of your estimates of standard deviation. If you recast your problem as comparing variances, then there are plenty of standard tests available.
For two independent samples, you can use the F test; its null distribution follows the (surprise) F distribution which is indexed by degrees of freedom, so it implicitly adjusts for what you're calling a scale problem. If you're comparing more than two samples, either Bartlett's or Levene's test might be suitable. Of course, these have the same problem as one-way ANOVA, they don't tell you which variances differ significantly. However, if, say, Bartlett's test did identify inhomogeneous variances, you could do follow-up pairwise comparisons with the F test and make a Bonferroni adjustment to maintain your experimentwise Type I error (alpha).
You can get details for all of this stuff in the NIST/SEMATECH e-Handbook of Statistical Methods.
@NRH's answer to this question gives a nice, simple proof of the biasedness of the sample standard deviation. Here I will explicitly calculate the expectation of the sample standard deviation (the original poster's second question) from a normally distributed sample, at which point the bias is clear.
The unbiased sample variance of a set of points $x_1, ..., x_n$ is
$$ s^{2} = \frac{1}{n-1} \sum_{i=1}^{n} (x_i - \overline{x})^2 $$
If the $x_i$'s are normally distributed, it is a fact that
$$ \frac{(n-1)s^2}{\sigma^2} \sim \chi^{2}_{n-1} $$
where $\sigma^2$ is the true variance. The $\chi^2_{k}$ distribution has probability density
$$ p(x) = \frac{(1/2)^{k/2}}{\Gamma(k/2)} x^{k/2 - 1}e^{-x/2} $$
using this we can derive the expected value of $s$;
$$ \begin{align} E(s) &= \sqrt{\frac{\sigma^2}{n-1}} E \left( \sqrt{\frac{s^2(n-1)}{\sigma^2}} \right) \\ &= \sqrt{\frac{\sigma^2}{n-1}} \int_{0}^{\infty} \sqrt{x} \frac{(1/2)^{(n-1)/2}}{\Gamma((n-1)/2)} x^{((n-1)/2) - 1}e^{-x/2} \ dx \end{align} $$
which follows from the definition of expected value and fact that $ \sqrt{\frac{s^2(n-1)}{\sigma^2}}$ is the square root of a $\chi^2$ distributed variable. The trick now is to rearrange terms so that the integrand becomes another $\chi^2$ density:
$$ \begin{align} E(s) &= \sqrt{\frac{\sigma^2}{n-1}} \int_{0}^{\infty} \frac{(1/2)^{(n-1)/2}}{\Gamma(\frac{n-1}{2})} x^{(n/2) - 1}e^{-x/2} \ dx \\ &= \sqrt{\frac{\sigma^2}{n-1}} \cdot \frac{ \Gamma(n/2) }{ \Gamma( \frac{n-1}{2} ) } \int_{0}^{\infty} \frac{(1/2)^{(n-1)/2}}{\Gamma(n/2)} x^{(n/2) - 1}e^{-x/2} \ dx \\ &= \sqrt{\frac{\sigma^2}{n-1}} \cdot \frac{ \Gamma(n/2) }{ \Gamma( \frac{n-1}{2} ) } \cdot \frac{ (1/2)^{(n-1)/2} }{ (1/2)^{n/2} } \underbrace{ \int_{0}^{\infty} \frac{(1/2)^{n/2}}{\Gamma(n/2)} x^{(n/2) - 1}e^{-x/2} \ dx}_{\chi^2_n \ {\rm density} } \end{align} $$
now we know the integrand the last line is equal to 1, since it is a $\chi^2_{n}$ density. Simplifying constants a bit gives
$$ E(s) = \sigma \cdot \sqrt{ \frac{2}{n-1} } \cdot \frac{ \Gamma(n/2) }{ \Gamma( \frac{n-1}{2} ) } $$
Therefore the bias of $s$ is
$$ \sigma - E(s) = \sigma \bigg(1 - \sqrt{ \frac{2}{n-1} } \cdot \frac{ \Gamma(n/2) }{ \Gamma( \frac{n-1}{2} ) } \bigg) \sim \frac{\sigma}{4 n} \>$$ as $n \to \infty$.
It's not hard to see that this bias is not 0 for any finite $n$, thus proving the sample standard deviation is biased. Below the bias is plot as a function of $n$ for $\sigma=1$ in red along with $1/4n$ in blue:
Best Answer
Let $X_1, ..., X_n \sim N(\mu, \sigma^2)$. As shown in this thread, the standard deviation of the sample standard deviation,
$$ s = \sqrt{ \frac{1}{n-1} \sum_{i=1}^{n} (X_i - \overline{X}) }, $$
is
$$ {\rm SD}(s) = \sqrt{ E \left( [E(s)- s]^2 \right) } = \sigma \sqrt{ 1 - \frac{2}{n-1} \cdot \left( \frac{ \Gamma(n/2) }{ \Gamma( \frac{n-1}{2} ) } \right)^2 } $$
where $\Gamma(\cdot)$ is the gamma function, $n$ is the sample size and $\overline{X} = \frac{1}{n} \sum_{i=1}^{n} X_i$ is the sample mean. Since $s$ is a consistent estimator of $\sigma$, this suggests replacing $\sigma$ with $s$ in the equation above to get a consistent estimator of ${\rm SD}(s)$.
If it is an unbiased estimator you seek, we see in this thread that $ E(s) = \sigma \cdot \sqrt{ \frac{2}{n-1} } \cdot \frac{ \Gamma(n/2) }{ \Gamma( \frac{n-1}{2} ) } $, which, by linearity of expectation, suggests
$$ s \cdot \sqrt{ \frac{n-1}{2} } \cdot \frac{\Gamma( \frac{n-1}{2} )}{ \Gamma(n/2) } $$
as an unbiased estimator of $\sigma$. All of this together with linearity of expectation gives an unbiased estimator of ${\rm SD}(s)$:
$$ s \cdot \frac{\Gamma( \frac{n-1}{2} )}{ \Gamma(n/2) } \cdot \sqrt{\frac{n-1}{2} - \left( \frac{ \Gamma(n/2) }{ \Gamma( \frac{n-1}{2} ) } \right)^2 } $$