Whats wrong with the singular value decomposition

linear algebramatricessvd

I am trying to calculate one by hand from Strang's "Linear Algebra and Its Applications…

$$
A = [-1, 2, 2]
$$

I can see the subspaces are…

$$
C(A) = [1]
\quad C(A^T) = \begin{bmatrix} -1 \\ 2 \\ 2 \end{bmatrix} \quad
N(A) = \begin{bmatrix} 2 \\ 1 \\ 0 \end{bmatrix},\begin{bmatrix} 2 \\ 0 \\ 1 \end{bmatrix} \quad
N(A^T) = \text{zero vector}
$$

from the rules

  1. first r columns of $U = C(A)$
  2. las m-r coluns of $U = N(A^T)$
  3. first r columns of $V = C(A^T)$
  4. last n-r columns of $V = N(A)$
  5. $\Sigma$ is the square root of the eigenvectors of $A^T A$

I get the SVD

$$
A =
\begin{bmatrix}
1
\end{bmatrix}
\begin{bmatrix}
3 & 0 & 0 \\
0 & 0 & 0 \\
0 & 0 & 0
\end{bmatrix}
\begin{bmatrix}
\frac{-1}{\sqrt5} & \frac{2}{\sqrt3} & \frac{2}{\sqrt3} \\
\frac{2}{\sqrt5} & \frac{1}{\sqrt3} & 0 \\
\frac{2}{\sqrt5} & 0 & \frac{-2}{\sqrt3} \\
\end{bmatrix}
$$

but numpy is telling me the $V^T$ matrix is

$$
\begin{bmatrix}
-\frac{1}{3} & \frac{2}{3} & \frac{2}{3} \\
\frac{2}{3} & \frac{2}{3} & -\frac{1}{3} \\
-\frac{2}{3} & -\frac{1}{3} & \frac{2}{3} \\
\end{bmatrix}
$$

So I must have done something totally wrong somewhere, but I can't see where

Best Answer

For a full SVD, a $1 \times 3$ matrix $A$ will decompose into $U \Sigma V^T$ with $U \in \mathbb{R}^{1 \times 1},\Sigma \in \mathbb{R}^{1 \times 3},V^T \in \mathbb{R}^{3 \times 3}$. (In general, for a full SVD, $\Sigma$ has the same shape as $A$, which means that $U$ and $V$ are square.) This means that you should not have the rows of zeros in $\Sigma$ even for a full SVD. Besides that, your $V$ is not orthogonal: you need to orthogonalize the subspace corresponding to the singular value of zero.

For a reduced SVD, a $1 \times 3$ matrix $A$ other than the zero matrix will decompose with $U \in \mathbb{R}^{1 \times 1},\Sigma \in \mathbb{R}^{1 \times 1},V^T \in \mathbb{R}^{1 \times 3}$. (In general, for a reduced SVD, $\Sigma$ is $r \times r$, which forces $U$ to be $m \times r$ and $V^T$ to be $r \times n$.) In your case this will just be $\begin{bmatrix} 1 \end{bmatrix} \cdot \begin{bmatrix} 3 \end{bmatrix} \cdot (A/3)$.

Note that you can, if you want, make a "bloated SVD", which in this situation would use the shapes $U \in \mathbb{R}^{1 \times 3},\Sigma \in \mathbb{R}^{3 \times 3},V^T \in \mathbb{R}^{3 \times 3}$. Then you would have your $\Sigma$ (including the rows of zeros), but your $U$ would need to be padded with zeros, and your $V^T$ would still need to be orthogonalized. There is really no reason, practical or theoretical, to use such a "bloated SVD", however.