This is a general question regarding multiple testing when making confidence intervals. Suppose I have an unknown vector
$$\mu = (\mu_1, \dots, \mu_p).$$
My goal is to estimate this vector, and create a $100(1-\alpha)\%$ confidence interval for each component $\mu_i$. I can of course make a $p$-dimensional ellipsoidal confidence region, but let's assume I don't want to do that.
Due to the problem of multiple intervals, my realized coverage probability is going to be smaller that $(1-\alpha)$, and so there is a need for correcting for this. The most common way is to use a Bonferroni correction, but Bonferroni is ridiculously conservative as $p$ increases.
My question: what is the least conservative method for correcting for multiple intervals?
–EDIT–
I am not assuming any model setup. Lets assume $\mu$ is the mean vector of a $p$ dimensional distribution $F$, which I sample from using, say Monte Carlo.
Best Answer
The least conservative multiple comparison adjustment is to not do one. In that case you are accepting a familywise error-rate that is (likely) larger than your per-comparison error-rate. How much larger depends on the correlation among your random variables. If you have perfect positive dependence between the coverage of any two intervals than your per comparison error rate is identical to your family-wise error rate:
$$P(\mu_1 \in CI_{u_1}, \mu_2 \in CI_{u_2}) = P(\mu_1 \in CI_{u_1}) = P(\mu_2 \in CI_{u_2}) $$
Complete independence between the coverage of any two intervals means that a joint interval of $p$ intervals will have coverage of $(1-\alpha)^p$:
$$P(\mu_1 \in CI_{u_1}, \mu_2 \in CI_{u_2}) = P(\mu_1 \in CI_{u_1}) * P(\mu_2 \in CI_{u_2}) $$
Bonferroni actually controls for perfect negative dependence between the coverage of any two intervals and so is the most strict:
$$P(\mu_1 \in CI_{u_1}, \mu_2 \in CI_{u_2}) = 1 - P(\mu_1 \notin CI_{u_1} \cup \mu_2 \notin CI_{u_2}) $$
If you do want to control for multiple comparisons, you have other options beside Bonferroni and beside controlling for the familywise error rate. One option is to instead control for the false discovery rate, which controls the proportion of false discoveries (or in your case, missed individual confidence interval) among your rejected hypothesis.
However, probably the best approach is to use a model that means you don't have to worry about multiple comparisons. In the classic linear modelling or ANOVA-like models, Andrew Gelman has pointed out that using multi-level models in large part frees us from worrying about multiple comparisons. In the context of your question, if you think that your various $\mu$'s are related to each other with positive or negative dependence, then use a multivariate probability model or a multi-level model, whichever is more appropriate for your conceptual model and research question.