Bonferroni Correction – Comparing Multiple Treatments

bonferronihypothesis testingmultiple-comparisonsrstatistical significance

I want to compare the impact of different drugs on different blood-parameters. We measure $m$ parameters. In our study we want to compare $n$ drugs with each other.

The blood-parameters for patients without drug administration, our healthy reference, are known and the same reference is used for all comparisons.

In the end we obtain a matrix with $n \times m$ dimensions filled with $p$-values, as we want to compare which parameters significantly change in one drug compared to another. My question is now how to best apply the Bonferroni correction in this setting.

The test persons in all $n$ groups are independent, no overlaps. I only have a dependency, as measurements are done on the same device and the reference is the same. Do I have to adjust the $p$-value per number of observations, or by the number of drugs AND observations? This means:

p-values * m
or
p-values * m * n

$m$: number of blood-parameters
$n$: number of drugs

Many thanks for your thoughts

Best Answer

To use the Bonferroni correction, you first have to decide how many hypotheses you are testing. That's the factor by which you divide your statistical significance threshold. If you are considering the hypothesis "drug A alters parameter X more than drug B does" for every possible pair of drugs A and B and for every possible parameter X, that's $mn(n-1)$ hypotheses in total. If $m$ and $n$ are large your significance threshold will be very stringent, so it might be better to focus on a smaller set of hypotheses, or not use the Bonferroni correction at all.