Interpret Odds Ratio – Understanding Odds Ratio in Fisher’s Exact Test

chi-squared-testfishers-exact-testoddsodds-ratior

I've tried using R operating Fisher's exact test to test for independence of the 2 datasets below and struggling to interpret the Fisher report. Please have a look at the data and the Fisher's test below.

Dataset 1: is a 2×3 contingency table

Consider then an experiment where we have 2 categorical variables: Group (I and II) & Outcome (Worse, Same, and Improve). These results might follow the administration of a new drug. We collect data and find out the frequencies as shown in the table below.

You can reproduce Dataset 1, and run the Fisher's test using the codes:

obs <- rbind(c(33, 44, 25),
             c(11, 28, 30))
rownames(obs) <- c("Group I", "Group II")
colnames(obs) <- c("Worse", "Same", "Improve")
obs

##          Worse Same Improve
## Group I     33   44      25
## Group II    11   28      30

fisher.test(obs)

##   Fisher's Exact Test for Count Data

## data:  obs
## p-value = 0.01049
## alternative hypothesis: two.sided

Dataset 2: is a 2×2 contingency table

Consider then an experiment where we have 2 categorical variables: Group (I and II) & Outcome (Worse and Improve). These results might follow the administration of a new drug. We collect data and find out the frequencies as shown in the table below.

vals <- matrix(c(3, 1, 3, 2),
               nrow = 2,
               dimnames = list(Group = c("Group I", "Group II"),
                   Outcome = c("Worsened", "Improved")))
##  Outcome
##  Group          Worsened Improved
##  Group I             3        3
##  Group II            1        2

fisher.test(vals)

## 
##  Fisher's Exact Test for Count Data
## 
## data:  vals
## p-value = 1
## alternative hypothesis: true odds ratio is not equal to 1
## 95 percent confidence interval:
##    0.06060903 156.52286969
## sample estimates:
## odds ratio 
##   1.852496

As you can see that I receive 2 different report templates for the two contingency tables a (2×3) and a (2×2). I have 3 questions:

Why the report of a 2×2 includes the Odds Ratio (OR) and the Confidence Interval, which I believe is a CI of the Odds Ratio, while the 2×3 doesn't have the OR in there?
Does that mean testing for independence using the value of OR can be applied ONLY for 2×2 contingency tables??
Could you help me interpret the Fisher's test report and point out the connection between the values of p-value, Odds ratio, and the Confidence Intervals?
In what case will we be using "one-sided Fisher's exact test"? What is the purpose of pointing out a Fisher's test should be one-sided or two-sided?
Can a table that's NOT 2×2 be tested using a "one-sided Fisher's exact test"?

Best Answer

The odds ratio cannot be defined for more than two levels of response. If you want an odds ratio, there are two options I can think of. One, convert the response into two levels by combining two of the levels. Two, ordinal regression.
For your second dataset, the point estimate of the odds ratio is greater than 1 and there is slightly more evidence that the odds ratio is greater than 1 compared to the evidence that it is less than 1. There is a smaller p-value if you specify the option alternative="greater". The option chosen should depend on the goals of the research.
Yes, if you use the ordinal regression, the hypothesis and p-value can be one-sided. For the general test of independence, no; it is a two-sided test. fisher.test allows options, but the p-value is identical in all 3 cases:

fisher.test(obs)  
fisher.test(obs, alternative="less")  
fisher.test(obs, alternative="greater")

You can also try

library(coin)  
cmh_test(as.table(obs))

The p-value is slightly different from fisher.test. That is because cmh_test uses an asymptotic p-value. If you use fisher.test(obs, hybrid=T), the p-value is closer to the asymptotic p-value from cmh_test.

Related Solutions

Solved – Fisher’s Exact Test, Relative Risk or Odds Ratio

Such guidelines exist, but either the odds ratio (OR) or the relative risk (RR) is fine. You only have to be sure that you interpret either one correctly. This is the advice given by Schmidt & Kohlmann (See below). Given that the relative risk is more intuitive, then it may be more advisable to use it. At least, this is the advice given by Deeks (See below). Deeks recommends the odds ratio only when it is obtained from case-control studies and logistic regression analysis.

Additionally, the CI would be much better at communicating the uncertainty about the effect than using a p-value. Both are analogous either way.

An additional outcome of interest for you could be the Number Needed to Treat, which is calculated from the absolute risk. In your situation using the 5% and 7% values you posted, one would need to treat ~50 people to reduce incidence by 1 person.

This tool calculates all 4 metrics given the 2x2 contingency table, and returns confidence intervals using a variety of methods. It makes use of the epitools R package. I'd be glad to answer any questions.

Schmidt, C. O., & Kohlmann, T. (2008). When to use the odds ratio or the relative risk? International Journal of Public Health, 53(3), 165–167. https://doi.org/10.1007/s00038-008-7068-3

Deeks, J. (1998). When can odds ratios mislead? Odds ratios should be used only in case-control studies and logistic regression analyses. BMJ (Clinical Research Ed.), 317(7166), 1155-6-7. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/9784470

Odds Ratio – Interpreting Zero Odds Ratio from Fisher’s Exact Test?

If in a 2-by-2 table, we let $n_{11}$ be the counts in (1,1) cell, $n_{12}$ the counts in the (1,2) cell, $n_{21}$ those of cell (2,1) cell and $n_{22}$ those of cell (2,2), then the sample odds ratio is defined by

$$ \hat\theta = \frac{n_{11}n_{22}}{n_{12}n_{21}}. $$

If any of $n_{ij}$ is zero, then $\hat\theta$ equals 0 or $\infty$ (it is undefined if both entries in a row or in a column are zero). Because these outcomes have positive probability, then the expected value and variance of $\hat\theta$ are not defined. A better well-behaved estimator is $$ \tilde\theta = \frac{(n_{11}+0.5)(n_{22}+0.5)}{(n_{12}+0.5)(n_{21}+0.5)} $$

Both $\hat\theta$ and $\tilde\theta$ have the same asymptotic normal distribution around the true odds-ratio $\theta$. Unless the sample size is quite large, however, the distributions of both $\tilde\theta$ and $\hat\theta$ are highly skewed. The log transform, having an additive rather than multiplicative structure, converges more rapidly to normality. An estimated standard error for $\log\hat\theta$ is

$$ \hat{\rm{se}}(\log\hat\theta) = \left(\frac{1}{n_{11}}+\frac{1}{n_{12}}+\frac{1}{n_{21}}+\frac{1}{n_{22}}\right)^{1/2}, $$

and thus by the large sample normality of $\log\hat\theta$

$$ \log\hat\theta \pm z_{\alpha/2}\hat{\rm{se}}(\log\hat\theta). $$ provides an approximate confidence interval for $\log\hat\theta$. The confidence interval for $\hat\theta$ can be obtained by exponentiating the limits of the above intervals. A similar interval can be obtained using $\tilde\theta$ in place of $\hat\theta$.

However, when $\hat\theta = 0$ or $\hat\theta = \infty$, this interval cannot be applied. When $\hat\theta = 0$, one should take 0 as the lower limit and when $\hat\theta = \infty$ one should take $\infty$ as the upper limit. For the other bound you can use the above formula following some adjustment, such as replacing $n_{ij}$ by $n_{ij}+0.5$ in the estimator and standard error. Other methods are also possible, such as methods based on score function or conditional maximum likelihood estimation.

As for the odd-ratio estimate provided by fisher.test, the help page (?fisher.test) says

estimate an estimate of the odds ratio. Note that the conditional Maximum Likelihood Estimate (MLE) rather than the unconditional MLE (the sample odds ratio) is used. Only present in the 2 x 2 case.

This means that the odd-ratio provided by fisher.test is derived through an alternative estimator (here conditional maximum likelihood, which is a kind of maximum likelihood estimator) which need not coincide with the sample odds ratio $\hat\theta$ or $\tilde\theta$.

Here is a simple example that supports this claim.

fisher.test(matrix(c(69,13,33,23), ncol=2))

    Fisher's Exact Test for Count Data

data:  matrix(c(69, 13, 33, 23), ncol = 2)
p-value = 0.001425
alternative hypothesis: true odds ratio is not equal to 1
95 percent confidence interval:
 1.556881 8.947718
sample estimates:
odds ratio 
   3.66139 

# sample odds-ratio estimate
> 69*23/(13*33)
[1] 3.699301

As you can see, the two estimates are not exactly the same although they are pretty close.

If you want to dive deeper into this subject I suggest having a look at Agresti's Categorical Data Analysis book.

Best Answer

Related Solutions

Solved – Fisher’s Exact Test, Relative Risk or Odds Ratio

Odds Ratio – Interpreting Zero Odds Ratio from Fisher’s Exact Test?

Related Question