What is the difference between varimax rotation and oblimin rotation in factor analysis?
Also, I am confused about the relationship between principal component analysis, varimax rotation and exploratory factor analysis, both in theory and in SPSS. How are they related?
Best Answer
You have several questions here. Let's start with the difference between PCA and Factor Analysis (FA). PCA gives you a transformation of the original variables into a new set that are mutually orthogonal. The first new component maximizes the variance. PCA answers the question, "What linear combination of my variables has largest variance? (subject to normalization of the weights)."
FA starts with a model about how the variables are related and where variation comes from in the data. This model implies that the covariance matrix will have a certain format. The key idea here is the latent variable (or factor). These factors are deemed to account for the interesting variation observed in the sample and the analysis attempts to retrieve them. Factor analysis can be done using various methods -- and one of these involves doing a PCA en route to the solution. But that's it. You don't actually need to do a PCA in order to get a FA.
Most confusingly, SPSS offers its PCA routine from the same analysis menu as exploratory factor analysis, thus encouraging in beginners the false idea that these methods are the same. The philosophy behind them is totally different.
Varimax and Oblimin. Factor analysis actually answers the question, "if my data, with its p variables say, actually comes from a q dimensional space (q < p) and the addition of noise, what is that q dimensional space?" In practice, the algorithms not only give you the q dimensional space, they also give you a basis for that space (these are the factors). But that basis may not be the best way to understand the q dimensional subspace. Factor rotation methods preserve the subspace and give you a different basis for it. Varimax returns factors that are orthogonal; Oblimin allows the factors to not be orthogonal.
Ideally, we would like factors that load "all or nothing" on the original variables ... as in "Questions 1 - 5 of the survey all relate to attitudes to authority; questions 6-10 all relate to a sense of justice". You want the factor coefficients to be large or 0. The rotation methods aim for that. The idea is to give you factors that are easier to interpret. Oblimin does a "better" job, given that it doesn't have to force the results to be orthogonal. On the other hand, the idea behind factors was that they account for the variation in the sample ... if the factors are correlated, what accounts for the relationship between the factors?
To me, I think it is probably best to got with Varimax during an exploratory FA. Then explore possible relationships between the factors in confirmatory factor analysis, which is better suited to that sort of modeling.
Note that SPSS as such does not do confirmatory FA or structural equation modeling. You need to purchase the Amos add-on for that. Alternatively, you can use the sem() or lavaan() functions in R.