I'm having trouble understanding the relationship between multiple and single linear regression. I have six variables $(x_1, \dots, x_6)$ I'm using in my model. If I check each one individually against $y$, all but $x_2$ come up significant (and four of those are $p < 0.001$). However, when all six are used in a multiple linear regression model, only two come up as significant.
This seems somewhat counter intuitive to me. I understand that multiple linear regression will look at how much each factor contributes to the overall model, but in the single regressions, it seemed like several variables were very important. Does it possibly have to do with scaling? In the single regressions, each variable produced a very different slope.
Best Answer
This makes total sense since your regressors aren't orthogonal to each other.
Refer to this wiki for a reference on collinearity:
http://en.wikipedia.org/wiki/Multicollinearity