Solved – Fisher’s method of combining p-values when one of the p-values is zero

combining-p-valueshypothesis testingp-value

I have a bunch of independent $p$-values and now I want to combine them using the Fisher's method. However, in R, when a $p$-value is zero, the log(p-value) becomes $-\infty$, so that the test statistic $X^2$ is $\infty$. This will give the Fisher's method $p$-value of 0.

If I check the $p$-values' histogram, they look pretty uniform, so I expect that the Fisher's method should give me a not-so-small $p$-value (should not reject the null). Is there a way to get around this issue? How about the Stouffer's method? Will that method automatically take care of the issue when $p = 0$ (or close to 0)?

Update: I tried the Stouffer's method, but still the issue of $\infty$ is not solved…

Best Answer

Irrespective of the discussion in the comments about how these $p$-values of $0$ arose there are methods for combining $p$-values which can be calculated if $p=0$.

As the OP indicated neither Fisher's method nor Stouffer's works.

The method of Edgington based on the sum of $p$, the closely related mean $p$ method, the method using logit of $p$, Tippett's method based on the minimum $p$ and variants of Wilkinson's method of which Tippett is a special case can all be calculated. Whether that is a sensible thing to do depends on the scientific question of course.

All the methods mentioned are available in the R package metap which, disclaimer, I wrote and maintain.