I have some data which in the terms of a 3×2 contingency table look in the following way.
Category 1 Category 2 Total
Example type 1 1140 60 1200
Example type 2 840 360 1200
Example type 3 1020 180 1200
Total 3000 600 3600
More specifically, I have data from 60 people all of which were shown 60 ambiguous stimuli (20 per example type). Each of those examples had to be categorized as Category 1 or Category 2.
I have several questions:
1) Is there a specific difference between the data represented in that way (3×2 contingency table) and in a 2×3 manner (where Category 1 and Category 2 are on the rows and Example type 1, 2 and 3 are as columns)?
2) If I am interested in whether there is an association between the two variables, is the chi-square test for association (independence) my best way to go?
3) Given that the survey is designed in this way – all people see 20 examples per row – is the assumption of independence violated?
4) How can I acquire more specific information about eventual interaction – for example, that people behave significantly different only for the Example type 2 cases?
Edit: the percentages in the table are changed to counts and the subtotals and totals are added.
Edit 2:
Thank you for your suggestion for using log-linear model.
I fitted the model as per the suggestion:
fit <- glm(Freq~category+example+category*example, data=dataDF, family=poisson)
The summary output is the following:
Deviance Residuals:
[1] 0 0 0 0 0 0
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) 7.03878 0.02962 237.657 < 2e-16 ***
categoryCategory 2 -2.94444 0.13245 -22.230 < 2e-16 ***
exampleExample type 2 -0.30538 0.04547 -6.716 1.87e-11 ***
exampleExample type 3 -0.11123 0.04310 -2.581 0.00986 **
categoryCategory2:exampleExample type 2 2.09714 0.14667 14.298 < 2e-16 ***
categoryCategory2:exampleExample type 3 1.20984 0.15518 7.797 6.36e-15 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
(Dispersion parameter for poisson family taken to be 1)
Null deviance: 2.0336e+03 on 5 degrees of freedom
Residual deviance: -7.7272e-14 on 0 degrees of freedom
AIC: 58.905
Number of Fisher Scoring iterations: 2
Do I understand correctly, that fitted in this way – if the Null deviance is low and the Residual deviance is high
this would indicate that the data do not plausibly
emenate from a logistic reegression model with a constant term only.
So, the correct way to extract a p-value from the Null deviance would be:
> 1 - pchisq(2033.6, 5)
> 0
and to extract a p-value from the Residual deviance would be:
> 1 - pchisq(0, 0)
> 1
We see that this p-value is large, meaning that it is plausible that the data emanate from a model which includes the IVs?
However, isn't it weird that all numbers (the first Deviance Residuals, the degrees of freedom, the residual deviance) as so small and so big?
Best Answer
As a previous comment pointed out, you need count, instead of the percentages, for these sort of analyses. What I answered below is based on count. It seems that you have 20 per row. So I assume that you indeed know the counts for each cell in the table.
No, they should be equivalent when testing associations between the rows and columns. There shouldn't be any difference.
You can certainly use Chi-square test for association. Another option is to use the G test. G test appears to perform better than Chi-square test when sample size is small (see here). More generally, you can fit a log-linear model to the data and test for independence. In fact, the G-test is the likelihood ratio test for independence in the log-linear model. In R, the model can be fit as glm(cell count~category+example+category*example, family=poisson). Testing if the interaction is significant is the test for independence. If you use likelihood ratio test, it will be the same as the G test. I think the log-linear model would be the best tool for this as it allows you to test other things in additional to independence.
No, it can be shown that Poisson sampling (i.e. each cell count is a Poisson random variable), multinomial sampling (i.e. total count of the table is fixed), or product multinomial sampling (row total or column total is fixed) are equivalent when testing the independence between row and column variables. So the log-linear model should apply in your experimental design. Chapter 3 of the categorical data analysis textbook by Agresti has a more detailed discussion on this if you wish to read in more depth about it.
I think the log-linear model would be suitable for this. After fitting the model, you can test differences between any two cell counts or any marginal means if you wish.
Edit based on the new information in the question:
The null deviance is the deviance that compares the model with only an intercept and the full model. In your case, the null deviance is huge, meaning that the data cannot be explained by just an intercept. The null deviance is testing that category and example and their interaction do not influence the count. In other words, the probability of all categories are the same, the probability in all examples are the same and consequently the probability in one category does not depend on example. This is not the hypothesis you want to test.
The model you fit is the full model, i.e. you fit a mean for each cell. Thus, you would expect deviance residual to be 0.
If you have any predictor in the model, the null deviance will always be greater or equal to the residual deviance. The null deviance tells you whether having predictors in the model gives you significantly better fit than just an intercept. The residual deviance give you ideas of whether the model has significant lack of fit (For grouped data, you can test lack of fit but for data that cannot be grouped, you cannot test lack of fit, but some suggests that you can still look at residual deviance as a hint of lack of fit).
You do not use null deviance to test for independence. Instead, you want to test if the interaction term, i.e. category*example, is significant. This can be done by anova(fit, test="Chisq") and looking at the p-value for the interaction term. Alternatively, you can fit a second model without interaction and compare it to the first model. This will achieve the same thing.