I want to use VIF to check the multicollinearity between some ordinal variables and continuous variables. When I put one variable as dependent and the other as independent, the regression gives one VIF value, and when I exchange these two, then the VIF is different. And once the VIF value is higher than 3, and the other time it is lesser than 3.
Then, how I do make a decision to keep the variable or not, and which one should I keep? Ultimately, I am going to use these variables in a logistic regression. How important it is to see multicollinearity in logistic regression?
Best Answer
It is important to address multicollinearity within all the explanatory variables, as there can be linear correlation between a group of variables (three or more) but none among all their possible pairs.
The threshold for discarding explanatory variables with the Variance Inflation Factor is subjective. Here is a recommendation from The Pennsylvania State University (2014):
Remember always sticking to the hypothesis previously formulated to investigate the relationship between the variables. Keep the predictors which make more sense in explaining the response variable.
Multicollinearity in logistic regression is equally important as other types of regression. See: Logistic Regression - Multicollinearity Concerns/Pitfalls.