Attempting to solve the SVD of this matrix:
\begin{bmatrix}
2 & 0 \\
0 & -3 \\
0 & 0
\end{bmatrix}
I know that one possible (my understanding is the SVD is not necessarily a unique result) result is:
$$
\begin{bmatrix}
1 & 0 & 0 \\
0 & -1 & 0 \\
0 & 0 & 1
\end{bmatrix}
\cdot
\begin{bmatrix}
2 & 0 \\
0 & 3 \\
0 & 0
\end{bmatrix}
\cdot
\begin{bmatrix}
1 & 0 \\
0 & 1
\end{bmatrix}^T
=
\begin{bmatrix}
2 & 0 \\
0 & -3 \\
0 & 0
\end{bmatrix}
$$
Which I have verified. I am trying to solve this problem myself, and am getting a different result; in particular, one that does not satisfy $A=U\Sigma V^T$. Therefore, I am trying to understand the hole in my approach.
Currently, my result looks like this:
$$
\begin{bmatrix}
0 & 0 & 1 \\
1 & 0 & 0 \\
0 & 1 & 0
\end{bmatrix}
\cdot
\begin{bmatrix}
3 & 0 \\
0 & 2 \\
0 & 0
\end{bmatrix}
\cdot
\begin{bmatrix}
0 & 1 \\
1 & 0
\end{bmatrix}^T
=
\begin{bmatrix}
0 & 0 \\
0 & 3 \\
2 & 0
\end{bmatrix}
$$
This is my approach, given a matrix $M$
1. Construct $MM^T$ and $M^TM$
$$
MM^T
=
\begin{bmatrix}
4 & 0 & 0 \\
0 & 9 & 0 \\
0 & 0 & 0
\end{bmatrix}
\quad\&\quad
M^TM
=
\begin{bmatrix}
4 & 0 \\
0 & 9 \\
\end{bmatrix}
$$
2. Solve for the eigenvalues and eigenvectors of both $MM^T$ and $M^TM$
This is accomplished via QR Factorization. Eigenvectors are listed in order of highest magnitude eigenvalue to lowest magnitude eigenvalue. The results are as follows:
$$
MM^T \enspace eigenvalues =
\begin{bmatrix}
9 & 4 & 0
\end{bmatrix}
$$
$$
MM^T \enspace eigenvectors =
\begin{bmatrix}
0 \\
1 \\
0
\end{bmatrix}
\begin{bmatrix}
0 \\
0 \\
1
\end{bmatrix}
\begin{bmatrix}
1 \\
0 \\
0
\end{bmatrix}
$$
$$
M^TM \enspace eigenvalues =
\begin{bmatrix}
9 & 4
\end{bmatrix}
$$
$$
M^TM \enspace eigenvectors =
\begin{bmatrix}
0 \\
1
\end{bmatrix}
\begin{bmatrix}
1 \\
0
\end{bmatrix}
$$
3. Construct $U$ using the eigenvalue-ordered eigenvectors of $MM^T$ as the columns of the matrix. Construct $V$ in the same way, but with $M^TM$.
I would also apply Gram-Schmidt orthonormalization here, but these matrices are already orthonormal.
$$
U =
\begin{bmatrix}
0 & 0 & 1 \\
1 & 0 & 0 \\
0 & 1 & 0
\end{bmatrix}
$$
$$
V =
\begin{bmatrix}
0 & 1 \\
1 & 0
\end{bmatrix}
$$
4. Finally, construct $\Sigma$ by making a diagonal matrix whose values are the square roots of the non-zero eigenvalues from $MM^T$ or $M^TM$.
Presumably, I should preserve the eigenvalue order when I do this?
$$
\Sigma =
\begin{bmatrix}
3 & 0 \\
0 & 2 \\
0 & 0
\end{bmatrix}
$$
So my question is, if anyone can aid me in identifying either the mistake I've made, or the hole in my approach to this problem. Assuming that I haven't made a simple error, I believe I am missing some sort of insight as to the construction of these matrices (because you can see, for instance, that the known solution has very similar values in differing orders and signs).
Best Answer
First, you slightly messed-up the order of the eigenvectors. For $MM^T$ is $u_1 =(0,1,0)$, $u_2 = c(1,0,0)$ and $u_3 = c(0,0,1)$ correspond to $9$, $4$ and $0$. For $M^T M$ the result is immediate with $v_1 = c(0,1)$ and $v_2 = (-1,0)$ correspond to $9$ and $4$.
Second, some more comments: It seems like you over-killed with the computations. Note that $MM^T$ is a diagonal matrix, hence its eigenvalues are the diagonal entries with orthogonal (normalized) eigenvectors. The non-zero eigenvalues of $M^TM$ are the same, while the eigenvectors (due to symmetry) are also the same up-to a sign. Namely, once you see $MM^T$ everything else is follows immediately without Gram-Schimdt, QR and so on. Regarding the choice of the signs - it is slightly more subtle, as the singular vectors are uniquely determined only up to a sign. As such you have to be careful with the sign choice in order to "reconstruct" the original $M$. Please see here Calculating SVD by hand: resolving sign ambiguities in the range vectors. the last answer about the choice of a sign.