These are statements I got from a pile of homework my colleagues have submitted for a statistics course
-
"…(if) mean is contained in the confidence interval, we can tell that there is a 95% probability of that the results are not significantly different from the population mean (null hypothesis)…"
-
"…CI tells you via raw data an interval where mean value is located at certain probability…"
I think these statements are wrong due to the fact that we cannot make probabilistic statements about means regarding confidence intervals.
For statement 1, it is not explicitly mentioned whether he is referring to the population mean, or the sample mean. Either way, I think he should've said "…with 95% confidence level that the result is not significantly different from the population mean…"
For statement 2, it is my understanding that neither population mean nor sample mean cannot be located at any interval at certain probability, since they're pretty much fixed values (for the sample mean, he's referring to the sample mean already acquired via data). To use the probability statement, I think he should've stated that "95% CI, once calculated, has 95% probability that it will include the population mean"
Best Answer
Number 1 seems totally incorrect to me. The sample mean is always in the confidence interval. If the population mean is inside the confidence interval, you made the right decision (i.e., assuming the confidence interval is being used as an interval estimate for the population mean). If the null value of the population mean (i.e., the proposed value of the population mean under the null hypothesis) is inside the confidence interval, then you don't have enough evidence to reject the null hypothesis. In all these scenarios, there is no probability statement to be made. Once the confidence interval is computed and a statement is made about some mean's (known) status as inside or outside the interval, no probability statements can be made. Events either occurred or didn't occur. Probability only refers to statements about tendencies over multiple trials, but there is only one trial here that has already been realized.
Number 2 is actually a little closer but is (as I understand it) closer to the interpretation of a Bayesian credible interval than a frequentist confidence interval. It's clear to me this author is talking about the population mean, which is what confidence intervals try to estimate. This is a common but incorrect interpretation of confidence intervals. Your statement "95% CI, once calculated, has 95% probability that it will include the population mean" is incorrect also, though. 95% CIs have a 95% probability of containing the population mean before they are calculated; once a sample has been drawn and a CI has been computed, the CI either contains or doesn't contain the population mean. There is no more probability involved because there is no random process to be realized; it has already been realized.
A correct statement about CIs is the following: in 95% percent of samples randomly drawn from a population, the computed 95% CI will contain the population mean. No probabilistic statements can be made about an individual confidence interval once computed. Although we don't know whether our computed CI contains the population mean, that uncertainty cannot be captured in a probability statement if we are operating under a frequentist perspective (which I assume you are, since confidence intervals are an inherently frequentist concept). The interval either contains or doesn't contain the population mean, and we don't know which is true. If we trust the CI-computing procedure, then we can be confident that our CI does contain the population mean, but that confidence is not associated with a true probability.