For a real symmetric matrix A, it's true that:
$$A A^T = A^T A = A^2$$
And since the right and left singular vectors of $A$ are the eigenvectors of $A^T A$ and $A A^T$ respectively, the right and left singular vectors ought to be identical.
Here's the particular situation that is causing my confusion. Let
$$A = \begin{bmatrix} 4 & 1 & -2 & 2 \\ 1 & 2 & 0 & 1 \\ -2 & 0 & 3 & -2 \\ 2 & 1 & -2 & -1\end{bmatrix}$$
Now when I try to take the SVD of this matrix, the matrices $U$ and $V$, of the left and right singular vectors respectively, are identical. Just as I thought they would be. In particular, for $U$, $\Sigma$, and $V$ I get:
$$U = V = \begin{bmatrix} -0.718 & 0.202 & 0.177 & -0.642 \\ -0.221 & 0.789 & 0.178 & 0.544 \\ 0.557 & 0.580 & -0.288 & -0.520 \\ -0.353 & 0.010 & -0.924 & 0.144 \end{bmatrix}$$
and
$$\Sigma = \begin{bmatrix} 6.845 & 0 & 0 & 0 \\ 0 & 2.269 & 0 & 0 \\ 0 & 0 & 2.198 & 0 \\ 0 & 0 & 0 & 1.084 \end{bmatrix}$$
But if I use this to calculate $U \Sigma V^T$ I don't get $A$. This already confuses me, but to make matters worse, if I take $V$ to be $U$ except with the signs changed on the third column, then it comes out correctly! I feel like I must be misunderstanding something about the SVD or Eigenvectors in general, as I thought that as long as each eigenvector had norm 1, it didn't matter what the sign of it was (i.e. I think about it as more of an eigendirection). Can anyone point out why I'm getting these results?
Best Answer
NOTE that while U == V, the matricies U and V are not symmetric. The definition of the SVD: A = USV' (where the prime indicates the Hermitian (conjugate transpose) of the matrix) - so this is probably your mistake - you probably forgot to apply the transpose - or your math language may have already supplied it for you.
In a lot of the languages with built-in SVD (ie Matlab, Python...) you need to be careful that the SVD function might return U,S,V' (the Hermitian of V)
for example in python you would have U,S,VH = svd(A)
This produces the same U,S,V as you have above - except that it gives the Hermitian of V
Putting it all back together... (in python 'dot' is used for matrix multiplication)
B = dot(U,dot(diag(S),VH))
yielding abs(B-A).max() = 9.992e-16, somewhere around machine epsilon