Benjamini-Hochberg Procedure – Understanding Rejection Threshold for Controlling False Discovery Rate

bonferronifalse-discovery-ratemultiple-comparisonsstatistical significance

Is there a possibility to calculate or estimate the overall rejection threshold of the Benjamini–Hochberg procedure (BH)?

For the correction of the FWER using the Bonferroni method, the significance threshold is adjusted to the number of evaluated hypotheses $m$ as follows $\bar{\alpha}= \frac{\alpha}{m}$. But since the BH-procedure produces an individual $q$-value for each independent hypothesis that is compared to an apriori defined FDR, I am not sure how this can be done.

Best Answer

As you sense, there is no fixed p-value cutoff for the Benjamini-Hochberg control of false discovery rate. The cutoff depends on the specific distribution of p-values among the $m$ hypotheses that you are evaluating together. You put them in increasing order and count up in $k$ from the lowest p-value $(k=1)$. You agree to "reject the null hypothesis" for hypotheses up through this value of $k$:

For a given $\alpha$, find the largest $k$ such that $P_{(k)} \leq \frac{k}{m} \alpha.$

If the null hypotheses all hold so there is a uniform distribution of p-values in [0,1], the p-value cutoff will be close to $\alpha$. How much below that you go if some null hypotheses don't hold depends on how non-uniform the distribution of p-values is.

Related Question