I heard just recently about PLS-DA and I was wondering how it differs from multinomial logistic regression, since logistic regression can be also used for categorical dependent variables.
Solved – What’s the difference between logistic regression and PLS-DA
logisticpartial least squaresregression
Related Solutions
PLS isn't necessarily a cure for insufficient sample size (though I recognize you're not claiming otherwise exactly). Check out Westland (2010) for a discussion of factors that determine the necessary sample size (and often determine that it's much larger than published articles often use). He discusses PLS too. One notable argument is that:
In PLS, the sample size question is probably both less relevant and less critical, because hypothesis testing is better left to LISREL and systems of equation approaches.
Some collinearity is generally expected among independent variables in basically any general(ized) linear model or structural equation model. It's mostly a problem when collinearity is very strong. Several methods exist for managing excessive collinearity, including ridge and principal components regression. Low collinearity is not a problem for any of these methods, including PLS. That they are somewhat more robust against high multicollinearity does not imply that they are biased by a lack thereof. Hence whether PLS is advisable in your circumstance probably depends on other factors.
You might try calculating the variance inflation factors for your set of regressors. These can give you a useful index of how much collinearity exists among them – it is often not as bad as one fears initially. This was the case in my own research: my concern about multicollinearity in my own regressions led me here to Cross Validated several months ago, incidentally.
Reference
Westland, C. J. (2010). Lower bounds on sample size in structural equation modeling. Electronic Commerce Research and Applications, 9(6), 476–487.
Yes. PLS-DA is basically PLS regression where Y consists of categorical variables. Here is an example of Y matrix with 3 groups each consists of 2 samples (the first row is headers and is not involved in calculations).
After applying PLS-DA you can obtain a BETA matrix (if you are using SIMPLS algorithm, for example) whose number of columns equals to the number of groups for 2+ groups. There are, however, few differences of PLS-DA from logistic regression and some other classification methods. The predicted Y values might get out of the 0 to 1 range. So you may find values such as 1.1 and -0.3 etc...
Another property of PLS-DA is the sum of the predicted Y values in row is always equal to 1.
Assigning predicted samples to a group can be done in several ways. The most common one is assignation of the sample to the group having the highest value. An alternative approach (Bayesian) is fitting a normal distribution to the predictions of the training set and finding the threshold that minimizes the classification error. The samples, then, can be assigned to the group whose threshold value is exceeded.
Best Answer
PLS-DA is closely related to LDA: for n > p the full rank PLS-DA (i.e. using all latent variables) is the same as LDA. For 1 latent variable, PLS-DA yields the same classification as closest (Euclidean) distance in feature space. I.e. the regularization "squeezes" the pooled covariance matrix into spherical shape.
A two class problem with both classes following a (multivariate) Gaussian distribution with the same covariance matrix (i.e. the situation where LDA is optimal), both LR and LDA yield the same solution.
LR will need more samples to get to the same stability, though.
In other words, there is a somewhat indirect relationship.
There are important differences between PLS-DA and LR in how they weight cases:
If you (ab)use PLS for dummy regression as it is frequently done in PLS-DA (i.e. y takes class labels encoded as 0 and 1 or equivalent encodings), PLS-DA will try to "squeeze" the within class distributions to points (as required in regression).
So if you want to use PLS for classification, make sure that it is appropriate to have all cases weighting in. Two situations where this is the case are