Solved – Post hoc Chi square non significant

chi-squared-testpost-hocr

I am interested to see if there is any relationship between eye color and whether someone is a vegan or not. The data is as follows:

      Vegan Non vegan
Blue     29        23
Green    15        32
Brown     3         9

I ran a Chi-squared test of association and the result is significant.

    Pearson's Chi-squared test

data:  vegan
X-squared = 7.4115, df = 2, p-value = 0.02458

However, when I ran the post-hoc of this, it turns out the p-values are not significant.

Adjusted p-values used the bonferroni method.

       comparison  raw.p  adj.p
1  Blue vs. Green 0.0255 0.0766
2  Blue vs. Brown 0.1069 0.3206
3 Green vs. Brown 0.7388 1.0000

What does this mean? If I look at the counts and expected counts of each group, I can see that people who are vegan tend to have blue eyes, and non-vegan tend to have green eyes.

R code you can try:

library(fifer)
vegan <-
    matrix(c(29,15,3,23,32,9),
           nrow = 3,
           dimnames =
           list(c("Blue", "Green", "Brown"),
                c("Vegan", "Non vegan")))
chisq.test(vegan)
chisq.post.hoc(vegan, control = "bonferroni")

Best Answer

It's due to the Bonferroni correction, which controls family-wise error rates. But, like all efforts to control that error rate, the cost is reduced statistical power. Applying the correction reduces the chance of a false positive and increases the chance of getting a false negative, which is consistent with what you have observed.

For interpretation, overall your data suggests that there is some statistically significant difference in number of vegans by eye color, but the pairwise comparisons of each category (using the Bonferroni correction) can't tell you which specific categories differ from each other.

Related Solutions

Solved – Post hoc chi-square test

Your chi-square test tells you that you have an interaction in your effects so the probability of subject interactions is dependent upon your dyad. Therefore, you're just checking to see which of those it's dependent upon and you can test. But you don't really need a test here... it's kinda obvious with the last row showing a 77:1 effect while the others are about or 3 or 4:1. If you want to do binomial test comparisons among the 3 perhaps just lower your alpha a little and don't try to conclude that the first two dyads are the same. You might also want to report a confidence interval for each individual dyad.

Another issue you have here is not one of post-hoc testing but one of the chi-square statistic being believed unreliable when the expected in a cell is less than 5 (for the male-male). However, this is probably not a problem.

Solved – Interpretation of post hoc chi square test results

Although the idea is there on the table in the OP, the best thing to do is to actually obtain the standardized residuals (as opposed to the residuals). This should be a straightforward choice in your software package. The formula for the standardized residuals is:

$$\text{Pearson's residuals}\,=\,\frac{{\color{red}{\text{O}}{\,\text{bserved}} - \color{blue}{\text E}\,\text{xpected}}}{ \sqrt{{\color{blue}{\text E}\,\text{xpected}}}}$$.

In this way you can establish a risk alpha of $0.05$, corresponding to a cutoff limit for statistical significance of $1.96$.

What you have in the output table right now is just the residuals: $\text{(observed - expected)}$.

Recapping:

Cell counts:

        Variable
Gender    1  2  3 Sum
  Male    9 21 34  64
  Female 27 26 17  70
  Sum    36 47 51 134

Expected counts:

        Variable
Gender    1  2  3 Sum
  Male   17 22 24  63
  Female 19 25 27  71
  Sum    36 47 51 134

Each cell in the OP contains the difference (without rounding).

Now compare to the standardized residuals:

        Variable
Gender      1    2    3
  Male   -2.0 -0.3  2.0
  Female  1.9  0.3 -1.9

They have the same sign as your initial values and have parallel relative magnitudes, but allow an immediate interpretation in the way that those with absolute value greater than $1.96$ can be considered statistically significant. In your case it seems as though column "1" and "3" may be responsible for a positive omnibus chi square test.

You can visually see this with a mosaic plot:

The standardized residuals are color-coded in blue for positive departure from expected values, and in red for negative deviation. In addition, the surface area covered by each cell is proportional to the cell count.

Code in R:

table <- matrix(c(1,21,34,35,26,17), nrow = 2, byrow = T)
dimnames(table) = list(Gender=c("Male","Female"), Variable=c("1","2","3"))
table; addmargins(table)
expect <- round(chisq.test(table)$expected)
addmargins(expect)
round(chisq.test(table)$residuals, 1)
library(vcd)
mosaic(table, shade=T, gp = shading_max)

Best Answer

Related Solutions

Solved – Post hoc chi-square test

Solved – Interpretation of post hoc chi square test results

Related Question