Solved – Fisher exact test on 4 x 3 table, with low count

categorical datafishers-exact-testr

I'm thinking about applying Fisher Exact Test for the following contingency table:

  Answer          Category-1  Category-2
  A-1             2           15
  A-2             0           17
  A-3             0           27
  A-4             2           15

However, I'm not sure if this would be correct for two reasons: 1. Some counts are zero. 2. Typically Fisher Exact Test is used on 2 x 2 tables, not sure if that's a restriction.

Also, I'm performing the test with the R command fisher.test

Best Answer

  1. Having observed zeros is not an issue for a Fisher Exact test -- nor indeed is it a problem for a chi-squared test (it's not clear why you think this would be a difficulty with an exact test; if you can clarify the source of your concern, additional explanation/clarification may be possible). An entire row or column of zeros might potentially be an issue for some implementations, though it's pretty easy to deal with if it arises (e.g. by combining with an adjacent row or column -- or equivalently, by eliminating it altogether - either way I think the outcome should be the same as including it in this case).

    One potential issue for you (not a problem for the test per se, but which may cause you concerns in some cases) is that entire columns or rows of low counts may in some cases lead to difficulty obtaining small p-values (the lowest attainable p-value may be greater than some common choices of significance level). This tends to be less of an issue with larger tables (I believe it doesn't pose a problem for the table in your question unless you require quite low significance levels)

  2. The test extends in a fairly natural way to $r\times c$ tables, using the likelihood (under the null) as a criterion by which to order the tables (effectively, likelihood is the test statistic; lower likelihood = "more extreme" in relation to calculation of p-values). The Fisher Exact test on an $r\times c$ table is sometimes referred to as a Fisher-Freeman-Halton test [1].

    Indeed, the help on the R function you mention makes it quite clear that it works on $r\times c$ tables, and gives several references that relate to the $r\times c$ case.

[1] Freeman, G.H., & Halton, J.H. (1951).
"Note on exact treatment of contingency, goodness-of-fit and other problems of significance."
Biometrika, 38, 141-149.

Related Question