confidence intervalp-value
I've been reading a lot on confidence intervals lately and I keep seeing statements such as: "A 95% confidence interval is a random interval that contains the true parameter 95% of the time" or "A confidence interval is a random variable because x-bar (its center) is a random variable."
Why is the confidence interval considered random? If it's truly random then why bother with confidence intervals at all? Am I missing something here?
If the bootstrapping procedure and the formation of the confidence interval were performed correctly, it means the same as any other confidence interval. From a frequentist perspective, a 95% CI implies that if the entire study were repeated identically ad infinitum, 95% of such confidence intervals formed in this manner will include the true value. Of course, in your study, or in any given individual study, the confidence interval either will include the true value or not, but you won't know which. To understand these ideas further, it may help you to read my answer here: Why does a 95% Confidence Interval (CI) not imply a 95% chance of containing the mean?
Regarding your further questions, the 'true value' refers to the actual parameter of the relevant population. (Samples don't have parameters, they have statistics; e.g., the sample mean, $\bar x$, is a sample statistic, but the population mean, $\mu$, is a population parameter.) As to how we know this, in practice we don't. You are correct that we are relying on some assumptions--we always are. If those assumptions are correct, it can be proven that the properties hold. This was the point of Efron's work back in the late 1970's and early 1980's, but the math is difficult for most people to follow. For a somewhat mathematical explanation of the bootstrap, see @StasK's answer here: Explaining to laypeople why bootstrapping works . For a quick demonstration short of the math, consider the following simulation using R:
# a function to perform bootstrapping
boot.mean.sampling.distribution = function(raw.data, B=1000){
# this function will take 1,000 (by default) bootsamples calculate the mean of
# each one, store it, & return the bootstrapped sampling distribution of the mean
boot.dist = vector(length=B) # this will store the means
N = length(raw.data) # this is the N from your data
for(i in 1:B){
boot.sample = sample(x=raw.data, size=N, replace=TRUE)
boot.dist[i] = mean(boot.sample)
}
boot.dist = sort(boot.dist)
return(boot.dist)
}
# simulate bootstrapped CI from a population w/ true mean = 0 on each pass through
# the loop, we will get a sample of data from the population, get the bootstrapped
# sampling distribution of the mean, & see if the population mean is included in the
# 95% confidence interval implied by that sampling distribution
set.seed(00) # this makes the simulation reproducible
includes = vector(length=1000) # this will store our results
for(i in 1:1000){
sim.data = rnorm(100, mean=0, sd=1)
boot.dist = boot.mean.sampling.distribution(raw.data=sim.data)
includes[i] = boot.dist[25]<0 & 0<boot.dist[976]
}
mean(includes) # this tells us the % of CIs that included the true mean
[1] 0.952
Look at $\chi$ distribution, it's a square root of $\chi^2$ distribution, which is in turn a sum of squared normals.
CORRECTION: $R(n)^2=\sum_{i=1}^n x_i+\sum_{i=1}^n y_i = \sum_{i=1}^{2n}z_i$, where $z_i=x_i$ for $i=(1,n)$ and $z_i=y_{i-n}$ for $i=(1+n,2n)$.
Hence, $R(n) = \sigma r(2n)$, where $r(k)\sim \chi(k)$.
$E[r(2n)]=\mu$, where $\mu=\sqrt{2}\Gamma((2n+1)/2)/\Gamma(n)$ The variance $Var[r(2n)]=(2n-\mu^2)$, see $\chi$ distribution. Subsequently, $E[R(n)]=\sigma E[r(2n)]$ and so on.
UPDATE: the CDF is given by the regularized gamma function: $P(n,r(2n)^2/(2))$. To compute the confidence bounds CB you have to solve for CB in $P(2n,CB^2/2)=\alpha$, where $\alpha$ is the confidence, such as 5% or 95%. CB will be in units of $\sigma$. Your math library should have the regularized gamma function, if it doesn't have its inverse then use the solver to find the CBs.
RESTATED problem
I think that it's best to redefine the $R(n)=\frac{1}{n}\sum_{i=1}^n r_i=\frac{1}{n}\sum_{i=1}^n\sqrt{X_i^2+Y_i^2}$. This means that you compute the distance $r_i$ for each pair of $(X_i,Y_i)$ coordinates, then average it acorss $n$ observation to get $R(n)$. Now, it's clear that $r_i^2\sim\chi^2_2$, assuming that X and Y are standardized normals, while $r_i\sim\chi_2$, i.e. chi distribution with 2 degrees of freedom. I gave the links to this distribution, you should be able to work out the math for non-standard normals.
Next, $R(n)$ is the sum of $\chi_2$ distributed numbers, so CLT should be applicable. For n=30 CLT should work great. I would run Monte Carlo then test it with Jarque-Bera or similar tests of normality for smaller n. If it's normal enough, then do the CLT for R(n), while working with closed-forms of $r_i$.
Example: $(\mu_X,\mu_Y)=(0,0)$, $\sigma_X=\sigma_Y=1$, $\sigma_{X,Y}=0$.
$E[R(1)]=E[r_1]=\mu=\sqrt{2}\frac{\Gamma(\frac{2+1}{2})}{\Gamma(1)}=1.2533$
$Var[R(1)]=Var[r_1]=2\cdot 1-\mu^2= 0.4292$
You can test this with the following Matlab/Octave code:
m=1e6 % number of samples
n=1 % number of X and Y to compute R
mu = 0*ones(2,1); % set ZERO means
Sig = eye(2); % set unit variance
x = mvnrnd(mu,Sig,m*n)'; % generate X,Y pairs
x = permute(reshape(x,2,n,m),[3,2,1]); % X and Y in 3 dim matrix
r = sqrt(sum(x.^2,3));
R = mean(r,2); % R(n)
hist(R) % show the histogram
jbtest(R) % test normality
[mean(R) std(R) var(R)]
Which outputs:
ans =
1.2519 0.6551 0.4291
Now you can run the same for higher n=30, and get the output:
ans =
1.2534 0.1197 0.0143
Applying the CLT approximation you get $Var[R(30)]_{CLT}=Var[R(1)]/30=0.0143$, very good match, and here's the histogram:
Best Answer
You just stated why in your question! You quoted this:
(In this case, it's presumably an interval for the mean, but the reasoning carries over to other confidence intervals.)
The sample mean is a statistic -- a quantity you calculate from the sample. Because random samples from some population are, well, random, things calculated from them are also going to be random.
Consider: If you drew a second sample from the same population would you have the same observations?
Would the sample mean be the same in both samples? Would the sample standard deviation be the same in both samples? The largest observation? The lower quartile?
No, they vary from sample to sample; indeed they're also random.
A confidence interval is also based on the random sample, so it, too, is a statistic (e.g. define it in terms of its endpoints) and it, too, is random.
Well presumably you'd like to use the data to calculate your interval. After all, it's the thing we have that tells us something about the population we drew the sample from.
If you're using the data - a random sample of your population - then useful quantities you calculate from it will also be random, including confidence intervals.
Random doesn't mean "ignores your data" -- for example a sample mean tells us about our population mean, and our sample standard deviation can be used to help us work out how far the sample mean will tend to be from the population mean.
In fact, we rely on the randomness - we exploit it to get the best possible use of information from our sample. Without random sampling, our intervals wouldn't necessarily tell us much of anything.
[You might like to ponder whether there might be a way to get an interval for a population quantity that is simultaneously reasonably informative and not random.]