Confidence Interval – Calculating and Interpreting Family-Wise Confidence Intervals

confidence interval

I have a bunch of variables organized into 10 different levels of a grouping factor. I'm doing some ANCOVA on particular variables and also plotting the data using boxplots. I'd like to add 84% confidence intervals to all the groups (since non-overlapping 84% CIs indicate a significant difference at alpha .05 – at least for two groups). I can do all this quite easily in R.

My question is – should I be applying a "family-wise" 84% CIs to all the groups? In other words, just as one would devalue an alpha level by the number of groups to obtain a family-wise alpha, should I inflate the CI a reciprocal amount to achieve a family-wise interval? This seems reasonable to me, but I haven't seen this discussed in the literature.

If alpha were CI were interchangeable for two or more groups the the family-wise 84% CI would be 99.5%, but i've read that alpha and CI are only interchangable for 1-sample situations. If this is the case, how would I go about calculating the family-wise confidence intervals for 10 (or any number) groups?

Any advice would be welcome.

best,

Steve

Best Answer

It sounds a reasonable solution if this is what important for you to present in the plot.

What this will give you (besides many questions, in case you are working with people who like statistics less then you), is a CI that is applicable to your situation which requires correction for multiple hypothesis.

What this won't give you, is the ability to compare difference between groups based on the CI.

Regarding the computation of the CI, you could use the p.adjust with something like simes which will still keep your FWE (family wise error), but will give you a wider window.

As to why you didn't find people writing about this, that is a good question, I don't know.