Random sampling, with and without replacement: difference between MSEs

probabilitystatistical-inference

In statistics, a simple random sample is a subset of individuals chosen (one by one) from a population. Each individual is chosen randomly such that each individual has the same probability of being chosen at any stage during the sampling process, and each subset of $k$ individuals has the same probability of being chosen for the sample as any other subset of $k$ individuals.
From a population of size $N$ with finite variance, a simple random sample of size $n$ is drawn without replacement, and a real-valued characteristic $X$ measured to yield observation $X_j$ $(j = 1,2,3, \ldots,n)$.

(a) show that the sample mean $\overline X_n$ is an unbiased estimator of the population mean $m$.

(b) show that the expected squared error of $\overline X_n$ as an estimator of $m$ is smaller than that of the mean of a simple random sample of the same size $n$ drawn with replacement.

(c) show that as $n,N \to \infty$ and $r=\frac{n}{N}$ and the population variance is always less than $M$ for all $N$, the difference between the expected squared errors of the two estimators is $O(r)$

For the question (b), I worked out the expected squared error of $\overline X_n$ is $\frac{N-n}{N-1}\frac{\sigma^2}{n}$. The other estimator with replacement is $\frac{\sigma^2}{n}$.

As to (c), the difference between them is $O\left(\frac{1}{N}\right)$. I wonder where I make the mistake? Thanks a lot.

Best Answer

Yep, the formulation of the question is strange : https://web.ma.utexas.edu/users/parker/sampling/woreplshort.htm

Your formulae are correct, the difference between the two is $$ \frac{\sigma^2}{rN}\left(\frac{N-rN}{N-1} - 1\right) \sim_{n,N\to\infty} -\frac{\sigma^2}{N} $$

The relative error expressed in % is of order $r$ though : $$ \frac{\frac{\sigma^2}{rN}\left(\frac{N-rN}{N-1} - 1\right)}{\frac{\sigma^2}{rN}} = \frac{1-rN}{N-1}\sim_{n,N\to \infty} -r $$

Related Question