The standard error of the mean is often calculated with the formula $\sqrt{\sigma^2 / n}$, which uses population variance (or an estimate of it).
However, standard error of the mean is defined as the standard deviation of the sample means. So why don't we ever directly calculate standard error of the mean as $\sqrt{E[(X_i – \overline X_n)^2]}$ where $X_i$ is a sample mean and $\mu$ is the mean of the sample means? Since the formula for standard deviation is $\sqrt{E[(X – \mu)^2]}$.
Best Answer
Well, typically, we only have a single realization of the sample mean to work with - we are not often in a situation where we have several datasets drawn from the same population. Even if we did have multiple samples, why not just combine them to get an even more precise estimate of the sample mean, given that we likely care more about estimating the quantity itself?
However, resampling techniques use this approach to estimate standard errors. The most familiar is likely the bootstrap procedure, which can be used to estimate the standard error of any statistic. If you are unfamiliar, bootstrapping returns many resampled sample statistics, which we take the standard deviation of to estimate the standard error of that statistic.