I know for PCA, it's true that the first N eigenvectors have N greatest variance.
But I'm not sure whether that's also true for NMF(Non-negative Matrix Factorization). For example, this method(Standard Nonnegative Matrix Factorization (NMF) [Lee2001], [Lee1999].): http://nimfa.biolab.si/nimfa.methods.factorization.nmf.html
And are there articles that write about how to calculate the variance of first N eigenvectors? What is the best k(k means counts of eigenvector) for dimension reduction?
Thanks!
Best Answer
Firegun - NMF and PCA are very different in theory, although their models and results are notationally v/similar (i.e., X = AB). NMF aims to decompose X to sparse factors that can explain it (while the data and factors are all positive), whereas PCA does the decomposition based on the amount of variance that a given factor/PC can explain. Thus, in the latter, variance-sorting is a sensible way of scoring components, whereas in NMF, it might not necessarily be so ... If you would like to use NMF for dimensionality reduction, the best way might be to look at the factors and see which ones are irrelevant to your signal (say, are noise) and zero their contribution (again, you still can do variance scoring, but if you think that variance scoring will define the importance, you might as well use PCA).