Confidence interval (CI) of the true value of a parameter is estimated using a sample. The interpretation of that CI is that many (say 100) such samples are taken, about 95% of the time the confidence interval formed from those samples would contain the true parameter value. A variation of this definition can be seen. This link has some answers.
I was wondering if there is any use of such a definition given that I would have to repeat the sampling/experiment many times (say 100 times). The explanation I settled to is that if a random and highly representative sample is taken/obtained, the CI that is formed from it would be a representative of those many samples (say 100). I cannot say the true value will be in the middle of that CI since true value is unknown.
But what else can we say? What is the use of the one CI I have constructed?
Best Answer
A confidence interval is typically more useful than a hypothesis test. A hypothesis test tells you if you can rule out a specific null hypothesis (typically, $0$). On the other hand a confidence interval demarcates an infinite set of values that, if they had been your null, would have been rejected similarly. (Likewise, it gives the set of potential null values that would not have been rejected.) For example, consider a 95% confidence interval for a mean $(.1, .9)$. The p-value for the (nil) null is $<.05$, but the confidence interval also lets you know that if your null value had been $1.0$, it would have been rejected as well.
A confidence interval also helps you differentiate between high level of confidence and a large effect. People are often impressed by an effect that is highly significant (e.g., $p<.0001$), and conclude that it must be really important. However, p-values conflate the size of the effect with the clarity of the effect. You can get a low p-value because the effect is large or because the effect is small, but you have very many data. This isn't ambiguous if you're looking at a confidence interval that is, say, $(.05, .15)$ versus $(5, 15)$.
In addition, a confidence interval is usually more informative than a point estimate. Although the point estimate returned by some fitting function will typically be the single most likely value (conditional on your data and your model), it isn't actually very likely to be the true value. There is, as you mention, no guarantee that the true value lies within a, say, 95% confidence interval (for instance, there isn't a 95% chance that the true value is in a 95% confidence interval1). That said, it is more likely that the true value lies within the interval than it is that the point estimate is the true value—this should be obvious since the point estimate is within the interval. In fact, you could think of a point estimate as a $0\%$ confidence interval.
1. Why does a 95% Confidence Interval (CI) not imply a 95% chance of containing the mean?