Solved – Assumptions and interpretation of Fisher’s Exact Test

categorical datacontingency tablesfishers-exact-testmeanr

I have this matrix:

x <- c(2, 38, 196, 2)

contingency <- matrix(x, nrow = 2, byrow = TRUE)

print(contingency)

     [,1] [,2]
[1,]    2   38
[2,]  196    2

And I've carried out this Fisher's Exact Test:

fisher.test(contingency)

which outputs this:

    Fisher's Exact Test for Count Data

data:  contingency
p-value < 2.2e-16
alternative hypothesis: true odds ratio is not equal to 1
95 percent confidence interval:
 6.103516e-05 4.703333e-03
sample estimates:
 odds ratio
0.000701445

My questions are:

The values in the matrix (2, 38, 196, 2) are means. Is it ok to run a Fisher’s Exact Test on these data?

If I was to conclude that the proportions in each group are unlikely to be equal, would i be correct?

Best Answer

The values in the cell of these tables should not be means, they should be counts. Fisher's test is valid only for contingency tables.

Related Solutions

Solved – Comparing p-values for Fisher’s exact test and test of equal proportions

prop.test uses a Pearson chi-square test. This is an asymptotic test. It will be worst when you have small samples or get too near the tails. Fishers will always be "better" because it is an "exact" test that does not rely upon asymptotic arguments to obtain its p-values...rather, it computes all the ways the table could have come about and then finds the proportion that were as-or-more-extreme.

Practically, this will result in Fisher's being less "powerful" when it matters because Pearson's approximation is most wrong in exactly those cases.

I do not know why fisher.test should take so long. For sample sizes on the order of $10^7$, it should have dropped to approximate methods unless the events are really rare. Are they? An alternative might be binom.test which uses Fisher's and may swap algorithms when sample sizes get large and event rates are still common. That might speed things up. A MonteCarlo version might work, also.

In your case and for sample sizes this high and non-rare events, Fisher's and Pearson's should not disagree to any real extent but I'd request the continuity-correction on Pearson prop.test(..., correct=TRUE). Try your simulation with this option and see if there is a dime's worth of difference then.

Another option is Barnard's unconditional test which can be more powerful but which many people frown at (even Barnard) though their cited reasons are often esoteric. In any case, that is not likely to be faster than either Pearson or Fisher.

Solved – Why does Fisher’s exact test produce the same p-value regardless of the predictor variable

The sameness of the p-values may be an artifact of the number of replicates used in the simulation. You might add a B=5000 or B=10000 option to fisher.test.
I'm not seeing what you're seeing. For example if I compare Josh to Shari, I don't get the same p-value as if I compare 1032 to 1093 or 6744 to 6817 (p < 0.0001, p = 0.8, p = 0.01, respectively).
"There's no way that an association exists." I wonder if you are thinking about the hypothesis being tested correctly. For each word, the proportions seen in Josh or Shari vary some. I also wonder if you're relying too heavily on the p-value, and ignoring an effect size or looking at the odds ratios.

 Input =("
 Name    kek lawl  lel lmao lmfao  lol  lul  mdr ptdr rjlol rofl roflmao rotflmao topkek trolol
 Josh     3    3    0   22     1  227    0   25    0     0    0       0        0      0      0
 Shari   11    1    6  472    99 3629    9   98   27     1    7       2        2      1      2
 ")
 Matrix = as.matrix(read.table(textConnection(Input),
                   header=TRUE,
                   row.names=1))

Matrix

fisher.test(Matrix, simulate.p.value=TRUE, B=10000)

prop.table(Matrix, margin=1)

library(vcd)

assocstats(Matrix)

loddsratio(Matrix)

Best Answer

Related Solutions

Solved – Comparing p-values for Fisher’s exact test and test of equal proportions

Solved – Why does Fisher’s exact test produce the same p-value regardless of the predictor variable

Related Question