My study has me running several regression analyses to investigate my hypothesis, and I am struggling with the multiple comparisons problem. I have three separate metabolites as predictor variables, and four other covariates in each model. (Running each metabolite in separate models is typical for these metabolites.) I am running both logistic regression analyses and linear regression for each metabolite. The outcome variable is the same for the respective sets of regression analyses (it was dichotomized for logistic regression and left continuous for the linear regression).
I've been through every note I've ever taken, all my books, and a lot of the internet, and I'm still not sure about the Bonferroni correction. I know it's probably a simple problem, but I'll be grateful for any and all help with it. Thanks!
Best Answer
You may be disappointed or relieved to hear that the answer is it depends. If your study is a preliminary exploratory study intended to determine which of the metabolites are worthy of following up, then there is no need to perform any correction for multiple comparisons. To do so robs you of power to find a real effect. On the other hand, if your experiment is designed to give a decisive result that you will be declaring without confirmatory studies, then a correction for multiple comparisons will reduce the risk of you being embarrassed by a false positive finding.