Solved – One-sided Fisher’s exact test and its complement

contingency tablesfishers-exact-testhypothesis testingnon-inferiority

If a one-sided Fisher's exact test testing the null hypothesis that A is not superior to B has a p-value of 0.98, is it true that the Fisher's exact test p-value for the null hypothesis that B is not superior to A is 1-0.98 = 0.02?

The question arises from the paper at nejm.org/doi/full/10.1056/NEJMoa1100403 where the primary outcome had a one-sided p-value of 0.98, but when I calculated the P value with a one-sided FET, I got a p-value of 0.09. Therefore, wondering why the "other" one-sided FET wouldn't have a p-value of 1-0.09 = 0.91 (instead of 0.98). Would appreciate help in knowing where I'm going wrong.

The Stata command used to generate the p-value of 0.09 was:

 csi  2  7 2850 2854 , exact

                 |   Exposed   Unexposed  |      Total
-----------------+------------------------+------------
           Cases |         2           7  |          9
        Noncases |      2850        2854  |       5704
-----------------+------------------------+------------
           Total |      2852        2861  |       5713
                 |                        |
            Risk |  .0007013    .0024467  |   .0015754
                 |                        |
                 |      Point estimate    |    [95% Conf. Interval]
                 |------------------------+------------------------
 Risk difference |        -.0017454       |   -.0037999    .0003091 
      Risk ratio |         .2866159       |    .0595926    1.378504 
 Prev. frac. ex. |         .7133841       |   -.3785042    .9404074 
 Prev. frac. pop |         .3561301       |
                 +-------------------------------------------------
                                  1-sided Fisher's exact P = 0.0904
                                  2-sided Fisher's exact P = 0.1793

Best Answer

The particular table helps a lot. The Fisher's exact test assigns probabilities to tables with these particular marginals using the hypergeometric distribution. In this case, we're thinking of drawing 9 balls from an urn (the cases) with 2852 white balls (exposed) and 2861 black balls (not exposed). The number of white balls drawn is the count for exposed cases. The distribution is:

0     1     2     3     4     5     6     7     8     9 
0.002 0.018 0.071 0.165 0.247 0.246 0.163 0.070 0.017 0.002

The one-sided test in your output is giving the probability of 2 or fewer: 

0.002 + 0.018 + 0.071 = 0.0904

The one-sided test in the other direction would give the probability of 2 or more, which is 1 minus the probability of 0 or 1:

1 - (0.002 + 0.018) = 0.98

Note that the two-sided test is the probability of 0, 1, 2, 7, 8, or 9, which does come to 0.179.

So the p-values for the two one-tailed tests don't add to one, because they each include the particular observed value and the distribution is discrete.

Related Question