Solved – How are p-value and odds ratio confidence interval in fisher.test are related

fishers-exact-testodds-ratiorscipy

I aim to conduct Fisher exact test on 2×2 matrix. It took a while to understand difference in R and Python scipy.stats implementation. Now I see that p-values are calculated the same way. And it seems like it doesn't use odds ratio doing that!

I still don't get connection between p-value and odds ratio. As far as I understood googling, odds ratio is a parameter of hypergeometric distribution matrices with fixed sum for rows and columns have.

R returns as output p-value, odds ratio and its confidence interval. I can either accept H0 (odds ratio = 1) or reject (than according to 'alternative' parameter it can be >1 or <1) depending on significance level.

I've also read that H0 is rejected if CI for odds ratio contains 1.

I have no idea how these two checks are related! Is there in this case dependence between significance and confidence level? Does it work like CI=95 then SL=2,5? Should I check both criteria or one of them is a consequence of the other?

Does it work the same in Python? It counts odds ratio differently and returns no confidence interval. Is it enough to test only p-value?

Best Answer

  1. The Fisher's Exact Test does not compute the $p$-value from the odds ratio.
  2. The Fisher's Exact Test tests the independence of a 2x2 table. If a table is independent, the odds ratio is 1, so they are related. The odds ratio is technically a parameter, and a test thereof is logistic regression or Pearson's $\chi^2$-test. Pearson's test and Fisher's Exact Test are asymptotically consistent, meaning they arrive at the same conclusion in the long run.
  3. For odds ratios that have continuously support and their (Wald test) p-values and CIs, it is true that if the 95% CI contains 1, the $p$-value is > 0.05. This is true of the relation between Pearson's $\chi^2$ test and logistic regression: the log odds ratio and its standard error can be used to compute both values arithmetically. Not true of FET.
  4. The Fisher's Exact test considers as a support all permutations of the 2x2 table conditional on the marginal frequencies as you say. For each of those tables an odds ratio can be computed. The support, then, of odds ratios is not continuous.
  5. To obtain 95% CIs and p-values which agree, one must "invert the hypothesis test" so that the 95% CI has only 5% of the possible tables with larger odds ratios in the tails under the null hypothesis. This is the de facto way of computing 95% CIs for odds ratios from Fisher's Exact Test: questions about programming are technically off-topic for this site, but for reporting you should verify this yourself by computing the results in Python, R, and maybe even SAS or Stata.

So yes the two "checks" (tests) are related: $p<0.05$ should be true only when the 95% CI does not include 1. All these results apply for other $\alpha$ levels as well.

Related Question