Solved – version of multivariate multinomial logit

categorical datamultinomial-distributionmultivariate analysis

I'm working with a data set with 2-3 response variables and 7 predictor variables. All the variables are categorical. If there were just one response variable, I think a multinomial logit would be the right model, but there are 2 or 3. So my question is – is there a multivariate version of the multinomial logit?

I've looked at several books on categorical data, but haven't seen anything like this (mainly using Agresti 2002).

I have about 2000 observations, though I'll probably need to split it up into 2 or 3 data subsets to really see what's going on. One thing I was thinking about is converting it to counts and use a model for count data. I could also combine the 2-3 response vars into 1 categorical with a lot of categories, but I think that will lower the chances of anything showing up for any of the categories. I could also do 2-3 separate models, one for each variable, which is obviously not as good.

I might also be able to get rid of some of the predictors (I think 3 of the 7 have the most explanatory power). I'm not opposed to using machine learning methods, I've found some interesting stuff already with decision trees.

thanks,

-paul

Best Answer

Agresti 2007 discusses them. They're in chapter 9 and 10. The 2002 edition probably discusses them too, as @suncoolsu mentioned.

Agresti refers to the group of response variables as a cluster and discusses according analysis with marginal models, conditional models and generalized estimating equations.

Related Solutions

Solved – test multicollinearity for multinomial logit regression

Multicollinearity

The issue seems to be that colldiag() needs a model parameter, which is not appended to the model fit by default in multinom(). But if you set model = TRUE in the multinom call, the colldiag function should compute without errors.

require(perturb)
set.seed(123)
a <- sample(4, 100, TRUE)
c <- sample(4, 100, TRUE)
a <- as.ordered(a)
c <- as.factor(c)
colldiag(mod = multinom(c ~ a, model = TRUE), scale = FALSE, 
         center = FALSE, add.intercept = TRUE)

Output:

Condition
Index   Variance Decomposition Proportions
         intercept X    
1  1.000 0.002     0.141
2  7.255 0.998     0.859

In regards to the output of perturb, I don't have any good answer as to how to interpret the results. The only comment given in the vignette states that

"If collinearity is a serious problem in the data, then the estimates will be unstable and vary strongly."

Autocorrelation

The dwtest() from {lmtest} should work with multinom() to compute autocorrelation for you, though you will need to convert your factor to a numeric variable.

require(lmtest)
dwtest(multinom(as.integer(c) ~ a))

Output:

      Durbin-Watson test

data:  multinom(as.integer(c) ~ a)
DW = 1.7298, p-value = 0.08517
alternative hypothesis: true autocorrelation is greater than 0

Especially for users of the mlogit function from the {mlogit} package: multinom()from {lmtest} did not work for me, it says in my just downloaded package version that this function doesn't exist. I had to reshape my data from a form fit for the mlogit function to a dataframe form where each item has only one line (instead of the number of potential outcomes per item). Then I could run this code: require(lmtest) dwtest(as.numeric(outcome) ~ predictors, data=dat).

Solved – Multinomial logit regression with ordinal independent variables

Ordinal independent variables are always tricky. As far as I know, there are essentially four sorts of ways of dealing with them, the first two are within the multinomial logistic scheme: Treat them as categorical and ignore the ordinality or treat them as continuous and pretend they are interval. One way of ameliorating some of the concerns of the latter pretense is to choose several different ordinal schemes and see if it makes a substantial difference. The latter two are row column effects association models (see Agresti, p. 154-184) or nonmodel based methods (see Agresti, p 184-224).

Related Question

Solved – Maximum number of categorical predictors in multinomial (polytomous) logistic regression