I'm trying to intuitively understand the difference between SVD and eigendecomposition.
From my understanding, eigendecomposition seeks to describe a linear transformation as a sequence of three basic operations ($P^{-1}DP$) on a vector:
- Rotation of the coordinate system (change of basis): $P$
- Independent scaling along each basis vector (of the rotated system): $D$
- De-rotation of the coordinate system (undo change of basis): $P^{-1}$
But as far as I can see, SVD's goal is to do exactly the same thing, except that resulting decomposition is somehow different.
What, then, is the conceptual difference between the two?
For example:
- Is one of them more general than the other?
- Is either a special case of the other?
Note: I'm specifically looking for an intuitive explanation, not a mathematical one.
Wikipedia is already excellent at explaining the mathematical relationship between the two decompositions ("The right-singular vectors of M are eigenvectors of $M^*M$", for example), but it completely fails to give me any intuitive understanding of what is going on intuitively.
The best explanation I've found so far is this one, which is great, except it doesn't talk about eigendecompositions at all, which leaves me confused as to how SVD is any different from eigendecomposition in its goal.
Best Answer
Consider the eigendecomposition $A=P D P^{-1}$ and SVD $A=U \Sigma V^*$. Some key differences are as follows,