When doing a simple random sample to estimate population mean for some statistic, how do I know whether sampling happens with or without replacement?
It feels wrong to use replacement, because 1) my AP stats teacher never does that and 2) I might use someone's data twice in the average.
But on the other hand, the proof that the statistic is an unbiased estimator of the mean is $$E(X)=E(X_1)+\cdots +E(X_n)=\mu+\cdots+\mu$$ which implies $$E\left(\frac{X}{n}\right)=\frac{n\mu}{n}=\mu$$ But doesn't this assume that the $n$ statistics $X_i$ are independent of each other? And isn't that only true if we replace after each sample?
Best Answer
Linearity of expectation doesn't rely on independence.
It's only the variance that's affected. If you sample without replacement (as most - but not all - population sampling is done), it reduces the variance a little (at least it's little under the common situation where the sample is much smaller than the population; hence the rule of thumb about ignoring it when the sample is sufficiently small)
For simple random sampling without replacement, it's actually quite easy to work out the mean and variance from fairly simple reasoning.
Formulas for estimating means and proportions under sampling without replacement are readily found - for example, here.