In Mathematics for Machine Learning, the authors claim the following: "For symmetric matrices, the eigenvalue decomposition and the Singular Value Decomposition are one and the same, which follows from the Spectral Theorem".
How does this claim follow from the Spectral Theorem?
Best Answer
The claim is true for positive semi-definite matrices, but not quite true for all symmetric matrices.
The spectral theorem implies that a symmetric matrix $A$ can be written as $UDU^\top$ for an orthogonal matrix $U$ and diagonal matrix $D$. This is almost already a valid singular value decomposition, except the diagonal elements of $D$ might not be nonnegative. To turn this into a SVD, let $\tilde{D}$ be the same as $D$ except with negative entries flipped to be positive, and let $V$ have the same columns as $U$, except the $i$th column negated if the $i$th element of $D$ was negative. Then check that $UDU^\top = U\tilde{D} V^\top$.