bootstrapestimation
I would like to estimate the true distribution of the following data set by making use of bootstrap method.
age = (21,81,85,27,39,61,15,20,39,40,87,87,69,59,54,71,66,88,1,2)
The population size is 20.
Can you please help me with both theoretical information and as well as the R programming code.
I took interest in this question because I saw the word bootstrap and I have written books on the bootstrap. Also people often ask "How many bootstrap samples do I need to get a good Monte Carlo approximation to the bootstrap result?" My suggested answer to that question is to keep increasing the size until you get convergence. No one number fits all problems.
But that is apparently not that question you are asking. You seem to be asking what the original sample size needs to be for the bootstrap to work. First of all I do not agree with your premise. The basic nonparametric bootstrap assumes that the sample is taken at random from a population. So for any sample size $n$ the distribution for samples chosen at random is the sampling distribution assumed in bootstrapping. The bootstrap principle says that choosing a random sample of size $n$ from the population can be mimicked by choosing a bootstrap sample of size $n$ from the original sample. Whether or not the bootstrap principle holds does not depend on any individual sample "looking representative of the population". What it does depend on is what you are estimating and some properties of the population distribution (e.g., this works for sampling means with population distributions that have finite variances, but not when they have infinite variances). It will not work for estimating extremes regardless of the population distribution.
The theory of the bootstrap involves showing consistency of the estimate. So it can be shown in theory that it works for large samples. But it can also work in small samples. I have seen it work for classification error rate estimation particularly well in small sample sizes such as 20 for bivariate data.
Now if the sample size is very small---say 4---the bootstrap may not work just because the set of possible bootstrap samples is not rich enough. In my book or Peter Hall's book this issue of too small a sample size is discussed. But this number of distinct bootstrap samples gets large very quickly. So this is not an issue even for sample sizes as small as 8. You can take a look at these references:
In Information Theory the typical way to quantify how "close" one distribution to another is to use KL-divergence
Let's try to illustrate it with a highly skewed long-tail dataset - delays of plane arrivals in the Houston airport (from hflights package). Let $\hat \theta$ be the mean estimator. First, we find the sampling distribution of $\hat \theta$, and then the bootstrap distribution of $\hat \theta$
Here's the dataset:
The true mean is 7.09 min.
First, we do a certain number of samples to get the sampling distribution of $\hat \theta$, then we take one sample and take many bootstrap samples from it.
For example, let's take a look at two distributions with the sample size 100 and 5000 repetitions. We see visually that these distributions are quite apart, and the KL divergence is 0.48.
But when we increase the sample size to 1000, they start to converge (KL divergence is 0.11)
And when the sample size is 5000, they are very close (KL divergence is 0.01)
This, of course, depends on which bootstrap sample you get, but I believe you can see that the KL divergence goes down as we increase the sample size, and thus bootstrap distribution of $\hat \theta$ approaches sample distribution $\hat \theta$ in terms of KL Divergence. To be sure, you can try to do several bootstraps and take the average of the KL divergence.
Here's the R code of this experiment: https://gist.github.com/alexeygrigorev/0b97794aea78eee9d794
Best Answer
Bootstrap won't give you the "true" distribution of you variable of interest, but rather an approximation that might be helpful in estimating parameters of the true distribution.
The idea is very simple: you sample with replacement $N$ cases from your dataset of $N$ observations the same way as you sampled your data from the population. In R that would look like this:
or simpler but more "hacky" way:
By using empirical estimates on bootstrap samples you can obtain parameters of the distribution of your variable (e.g. mean, mode, variance).
As about references, check original paper by Efron (1979) and the two books referenced here. You can find further description of bootstrap in this thread: Explaining to laypeople why bootstrapping works