We've gone over this concept in my class a couple of times, and it still isn't clicking.
Could someone please prove this statement:
The ideal bootstrap standard error of the mean
$$se_\infty(\bar{x})$$
(The standard error of bootstrap resampled means, where the number of bootstrap samples $B=\infty$),
is equivalent to the usual estimated standard error of the mean:
$$se(\bar{x}) = \dfrac{\sqrt{\dfrac{1}{n}\sum_{i=1}^n (x_i – \bar{x})^2}}{\sqrt{n}} = \dfrac{s}{\sqrt{n}}$$
Please be as detailed as possible.
Note: Here is an explanation given by my TA, I'd like it if you could clarify his explanation:
The "ideal bootstrap estimate of the standard error" is the standard error of a statistic under the distribution $F_n$ where $n$ samples are drawn iid from the empirical distribution. The variance of the mean of $n$ such variables is $1/n$ the variance of a single variable (using general properties of the mean). The variance of a single variable is by definition just the variance of the empirical distribution, i.e., the empirical variance. This is the desired claim.
Best Answer
I don't know what "ideal" means here.
The expression you have for $se(\bar{X})$ is the method-of-moments estimator of the standard error of the mean (without correction for degrees of freedom), which is asymptotically correct but is biased if $X$ does not follow a normal distribution in which case that expression actually is the maximum likelihood estimator of the SD and the exact SE for an exact (not approximate) normal distribution.
So if we take "ideal" to mean that $X$ are iid normal RVs, then we just appeal to the linearity of the expectation and variance of independent RVs and that linear combinations of normals are normal to show that they're equal. Hint: for the variance of a bootstrap sample, you can think of it two ways: a pseudolikelihood where two identical objects in the same sample are simply taken to be independent, or you can use the law of total variance to include the variance of the bootstrap sample and combine estimates ala Rubin's Rules.