Solved – How to detect multicollinearity in a logistic regression where all the independent variables are categorical and binary

binary datacategorical datalogisticmulticollinearity

I'm doing a multivariate logistic regression where all my independent variables are categorical and binary. I have transformed all my categorical variables into dummies in order to have reference groups and interpret my odds-ratios. However, I would like to check if there are eventually multicollinearity issues.

I plan to calculate Spearman correlation between my independent variables and calculate VIF too. Nevertheless, I have several questions: Is Vif appropriate in this case? Do I have to calculate VIF and Spearman correlation on my categorical variables or their associated dummies?

I have tested other ways to detect multicollinearity (I check if the coefficients vary a lot if I increase my sample size or drop or add variables) and I'm pretty sure there is not a collinearity issue but I would like to have a "quantitative" proof.

Best Answer

Daryl Pregibon has written a key paper on LR diagnostics. It looks like it's only available from the publisher:

Daryl Pregibon. Logistic regression diagnostics. The Annals of Statistics, volume 9, pages 705–724, 1981

That said and from a practitioner's point of view, I would be comfortable using the metrics that have been developed for OLS regression such as the VIF or the eigenvalue-based collinearity index. The best source for this class of diagnostics is Belsey, Kuh and Welsch's book Regression Diagnostics: Identifying Influential Data and Sources of Collinearity:

http://www.amazon.com/Regression-Diagnostics-Identifying-Influential-Collinearity/dp/0471691178/ref=sr_1_sc_1?ie=UTF8&qid=1457527579&sr=8-1-spell&keywords=regression+diagnositcs+belsey

Of course, stringent purists or a PhD dissertation committee would likely object to leveraging these readily available and easily implemented methods but they stand as useful proxies for the real thing.

Related Solutions

Solved – How to test for and remedy multicollinearity in optimal scaling/ordinal regression with categorical IVs

If you are using R, SPSS or Stata, you can look at the perturb package. It diagnoses collinearity by adding random noise to continuous variables; for categorical variables, some are changed to different categories.

In the documentation for perturb in R, it notes that the model need not be lm, implying that any model (including ones built with optimal scaling or ordinal logistic) could be used.

Solved – How to test for multicollinearity among non-linearly related independent variables

Multicolinearity is all about the linear relationship among you independent/explanatory/right-hand-side/x-variables. That you want to use those variables in a non-linear model does not matter. The logic behind that is that if you want to add both variables to your model then you have te be able to distinguish between a unit change in one variable and a unit change in the other. If the variables are linearly related then a unit change in one coincides with $k$ units increase in the other variables, where $k$ is some constant, so we cannot determine the separate effects of both variables. If the relationship is non-linear a unit change in one variable coincides with a variable number of units change in the other, so we are able to distinguish between the variables. So if you graphically determined that there is a relationship but that relationship is non-linear then that fact alone has already solved most of your problems.

Consider the following example: if we add a quadratic curve, that is, we add a variable $x$ and a variable $x^2$ to our model, then the relationship between the variables $x$ and $x^2$ is extremely strong. Still we can estimate that model. The reason is that that relationship is non-linear.

I find it informative to see a situation where this can break. Consider we have a study where we want to consider year of birth, which ranges between 1950 and 1990. If we just add that and its square then you might get into trouble as the relationship between birthyear and birthyear$^2$ is almost linear, as you can see below. You can solve this by centering at a meaningful variable within the range of your data, e.g. 1960. As you can see in the second graph the relationship is now non-linear and that is usually enough to solve the problem.

enter image description here

I created that graph with Stata using the following code:

twoway function xsquare = x^2, range(1950 1990) ///
    name(a,replace) title(uncentered) ytitle("x{sup:2}")
twoway function xsquare = (x-1960)^2, range(1950 1990) ///
    name(b, replace) title(centered) ytitle("(x-1960){sup:2}")
graph combine a b, ysize(3)

Best Answer

Related Solutions

Solved – How to test for and remedy multicollinearity in optimal scaling/ordinal regression with categorical IVs

Solved – How to test for multicollinearity among non-linearly related independent variables

Related Question