I understand that the Singular Value Decomposition is defined as SVD = $U\Sigma V^T$ , but I am slightly confused about the calculations when the matrix is not square.
For example, I have the matrix:
$$
\begin{bmatrix}
1 & -1 \\
-2 & 2 \\
2 & -2
\end{bmatrix}
$$
When I am solving for $V$, however, I am missing the last component. Have I done something wrong when calculating for matrices that are not square matrices?
$$\det(A^T A – \lambda I) =
\begin{bmatrix}
2 – \lambda & -4 & 4 \\
-4 & 8 – \lambda & -8\\
4 & -8 & 8 – \lambda
\end{bmatrix}
$$
$\lambda = 0, 2, 16$
Eigenvectors respectively are:
\begin{bmatrix}
1 \\
1/2 \\
0
\end{bmatrix}
\begin{bmatrix}
1 \\
2/7 \\
-2/7
\end{bmatrix}
\begin{bmatrix}
0 \\
1 \\
-1
\end{bmatrix}
Therefore $$\Sigma =
\begin{bmatrix}
\sqrt 2 & 0 & 0 \\
0 & \sqrt 16 & 0 \\
0 & 0 & 0
\end{bmatrix}$$
Also, $$V =
\begin{bmatrix}
7/\sqrt 57 & 0 & 2/\sqrt 5 \\
2/\sqrt 57 & 1/\sqrt 2 & 1/\sqrt 5 \\
2/\sqrt 57 & -1/\sqrt 2 & 0
\end{bmatrix}$$
This is the portion I am confused about.
Is $U = AV / \sqrt\lambda $ ? What if I have am missing a vector so that I can only get the first two columns of $U$?
Best Answer
First, correct a simple error $$ \mathbf{A}^{*}\mathbf{A} = \left[ \begin{array}{rr} 9 & -9 \\ -9 & 9 \\ \end{array} \right]. $$
This system matrix $\mathbf{A}$ is a rank one matrix (column 2 = $-$column 1) with singular value decomposition $$ \begin{align} \mathbf{A} &= \mathbf{U}\, \Sigma \, \mathbf{V}^{*} \\ % \left[ \begin{array}{rr} 1 & -1 \\ -2 & 2 \\ 2 & -2 \\ \end{array} \right] &= % U \left[ \begin{array}{ccc} \frac{1}{3} \color{blue}{\left[ \begin{array}{r} -1 \\ 2 \\ -2 \end{array} \right]} & \frac{1}{\sqrt{5}} \color{red}{\left[ \begin{array}{r} -2 \\ 0 \\ 1 \end{array} \right]} & \frac{1}{3\sqrt{5}} \color{red}{\left[ \begin{array}{r} 2 \\ 5 \\ 4 \end{array} \right]} % \end{array} \right] % sigma \left[ \begin{array}{cc} 3 \sqrt{2} & 0 \\ 0 & 0 \\ 0 & 0 \\ \end{array} \right] % V \frac{1}{\sqrt{2}} \left[ \begin{array}{rc} \color{blue}{-1} & \color{blue}{1} \\ \color{red}{1} & \color{red}{1} \\ \end{array} \right]. \end{align} $$ Blue vectors are in range spaces, red vectors in null spaces.
The thin SVD uses the range space components only: $$ \mathbf{A} = % U \frac{1}{3} \color{blue}{\left[ \begin{array}{r} -1 \\ 2 \\ -2 \end{array} \right]} % S \left( 3\sqrt{2} \right) % V \frac{1}{\sqrt{2}} \left[ \begin{array}{rc} \color{blue}{-1} & \color{blue}{1} \end{array} \right]. $$
You may benefit from this example: SVD and the columns — I did this wrong but it seems that it still works, why?