categorical datainterpretationlogisticregression
I am trying to interpret categorical variables with more than two classes. Some are significant whilst other classes are not. What can I infer from the insignificant ones? Does this mean the insignificant ones and the reference category equally influence the dependent variable?
For example:
ETHNICITY (Reference Category - Indian)
Other Asian: Sig = .273 exp(b) = 1.123
African: Sig = .000 exp(b) = .148
Generally, you should start from the highest order interactions. You are probably aware that it is usually not sensible to interpret a main effect A when that effect is also involved in an interaction A:B. This is because the interaction tells you that the effect of A actually depends on the level of B, rendering any simple main effect interpretation of A impossible. In the same way, if you have factors A, B, C, then A:B should not be interpreted if A:B:C is significant.
Thus, when you have a 5-way interaction, none of the lower-order interactions can be sensibly interpreted. Therefore, if I understand you correctly and you have interpreted your lower order interactions, you should probably not continue along those lines.
Rather, what you can do is to split up your data set and continue to analyze factor levels of your data set separately. Which of the factors you use to split up the dataset is arbitrary, but often it is very useful to split up the data for each variable and assess what you see. In your example, you might start with sex, and calculate an ANOVA for males, and another one for females (each ANOVA contains the 4 remaining factors). Just as well, you could split up the data according to ethnicity (one ANOVA for Asian, one for Caucasian). You could also split up by one of the within-subject factors.
I will assume that you have decided to split the data by sex (just to continue with the example here). Then, assume that for males, you get a 4-way interaction. You would then go on to split up the male data by one of the remaining variables (say, ethnicity). You would then calculate ANOVAs for male Asians (over the remaining 3 factors), and for male Caucasians.
Importantly, if you get only a lower-order interaction, then you are only "allowed" to analyze these further. This is because the other factors did not show significant differences. Thus, if your males ANOVA gives you only a 2-way interaction, then you would average over the other factors and calculate only an ANOVA over the 2 interacting factors (and, because we are in the male part of the ANOVAs, this would be for the males alone).
For the females, everything may look different, and so the decision which follow-up ANOVAs to calculate is separate for this group. So, what you did for males should be done for females in the same way ONLY if you got the same interactions.
Thus, you will potentially have a lot of ANOVAs, and it might not be easy to decide which ones to report. You should report 1 complete line down from the hightest interaction to the last effects (possibly t-tests to compare only 1 of your factors at the end). You should not usually report several lines (e.g., one starting the split-up by sex, then another one starting by ethnicity). However, you must report a complete line, and cannot simply choose to report only some of the ANOVAs of that line. So, you report one complete analysis, not more, not less. Which way to go in terms of splitting up / follow-up ANOVA is a subjective decision (unless you have clear hypotheses you can follow), and might depend on which results can be understood best etc.
Since you are interested in ranking the categories, you may want to re-code the categorical variables into a number of separate binary variables.
Example: Create a binary variable for express delivery- which would take the value 1 for express delivery cases and 0 otherwise. Similarly, a binary variable for standard delivery.
For each of these recoded binary variables you can calculate the marginal effects as indicated below:
Let me explain a bit on the above equation: lets say d is the re-coded binary variable for express delivery
is the probability of event evaluated at mean when d=1
is the probability of event evaluated at mean when d=0
Once you calculate the marginal effects for all the categories (re-coded binary variables) you can rank them.
Best Answer
In general, you do not want to interpret the p-values for the levels of a categorical variable that come with typical statistical output. By default, categorical values are represented in a model (logistic regression or otherwise) using reference level coding1. The tests in your statistical output are comparing each of the non-reference categories to the reference category. It is quite possible for none of those to be significant, but for there to be significant differences amongst the non-reference categories. To get a meaningful test of a categorical variable, you want to drop all levels of the categorical variable (i.e., the whole categorical variable) from the model and perform a nested model test2. Note that the reduced model could be a null (intercept only) model and that a nested model test for a logistic regression would be a likelihood ratio test instead of an $F$-test.
If you believe, based in part on the result of the nested model test, that the categorical variable is relevant to the model, then you can try to determine which levels differ. This is analogous to determining which groups differ following a one-way ANOVA. Bear in mind that a non-significant difference between two levels does not mean those levels are the same3 with respect to how they influence the dependent variable.
1. To better understand reference cell coding, my answer here may help: Regression based for example on days of week.
2. I explain nested model tests here: Testing for moderation with continuous vs. categorical moderators.
3. For more on that idea, it may help you to read my answer here: Why do statisticians say a non-significant result means "you can't reject the null", as opposed to accepting the null hypothesis?