I used PCA to reduce the features of 1020 vector size into desired length like 32, 64, 128 etc.
Now I want to make LDA dimensionality reduction to compare them.
I'm stuck now can't figure out how do I reduce into desired feature vector length the original data set using LDA. (I want it to reduce it to 32, 64 etc).
I'm using the following MATLAB code to achieve it:
LDA
The problem is that I have 5 classes and that code reduces the features size into 4. I tried to see what the code does but it seems the maximum what I can get is the feature size of 5.
Best Answer
In LDA, unlike PCA, the maximum number of features after dimensionality reduction is $c - 1$, where $c$ is the number of classes. Since you have 5 classes, the maximum number of features is 4. You can learn the math here. On slide 23, the maximum rank of the between class scatter matrix is $S_B$ is $c-1$. So, the maximum number of eigenvalues/eigenvectors is $c-1$. This is why the maximum number of features is $c-1$.