If $(U,\Sigma,V)$ is a singular value decomposition of $A$, do the first $\text{rank}A$ columns of $V$ and $U$ form orthonormal bases

eigenvalues-eigenvectorshilbert-spacesmatricesnumerical linear algebrasvd

Let

  • $m,n\in\mathbb N$
  • $A\in\mathbb R^{m\times n}$ and $|A|:=\sqrt{A^TA}$
  • $r:=\operatorname{rank}A$
  • $\sigma_1>\cdots>\sigma_r>\sigma_{r+1}=\cdots=\sigma_n=0$ denote the singular values of $A$

We say that $(U,\Sigma,V)$ is a singular value decomposition of $A$ if

  1. $U\in\mathbb R^{m\times n}$ is a partial isometry;
  2. $\Sigma=\operatorname{diag}(\sigma_1,\ldots,\sigma_n)\in\mathbb R^{n\times n}$;
  3. $V\in\mathbb R^{n\times n}$ is orthogonal

and $$A=U\Sigma V^T\tag1.$$

Can we show that

  1. The first $r$ columns $(e_1,\ldots,e_r)$ and $(f_1,\ldots,f_r)$ of $V$ and $U$ are orthonormal bases of $\mathcal R(|A|)$ and $\mathcal R(A)$, respectively?
  2. $\mathcal N(U)=\mathcal N(A)$ (noting that $\mathcal N(A)=\mathcal N(|A|)$)?

Best Answer

You have $$\tag1 AA^*=U\Sigma^2 U^*.$$ Taking the trace in $(1)$, $$ \operatorname{Tr}(\Sigma^2)=\operatorname{Tr}(AA^*)=\operatorname{Tr}(U\Sigma^2U^*)=\operatorname{Tr}(\Sigma^2 U^*U). $$ Then $$ 0=\operatorname{Tr}(\Sigma^2\,(I-U^*U))=\operatorname{Tr}(\Sigma\,(I-U^*U)^2\,\Sigma) $$ As the trace is faithful, we get $\Sigma(I-U^*U)^2\Sigma=0$, and so $(I-U^*U)\Sigma=0$. So $$\tag2 \Sigma=U^*U\Sigma=\Sigma\,U^*U. $$ Now $$ A^*A=V\Sigma U^*U\Sigma V^*=V\Sigma^2\,V^*. $$ Now $$ \ker A=\ker A^*A=\ker V\Sigma^2 V^*=\ker \Sigma V^*. $$ So, taking orthogonals,
$$ \operatorname{ran} A^*=\operatorname{ran}V\Sigma. $$ This shows that the first $r$ columns of $V$ span the range of $A^*$ (which is the same as the range of $|A|$). Going back to $(1)$, $$ \ker A^*=\ker AA^*=\ker U\Sigma^2\,U^*=\ker \Sigma U^*, $$ so $$ \operatorname{ran} A=\operatorname{ran} U\Sigma, $$ so the first $r$ columns of $U$ span the range of $A$.

It is not true in general that $\ker A=\ker U$. For instance take $$ A=\begin{bmatrix} 0&0\\1&0\end{bmatrix} \,\begin{bmatrix} 1&0\\0&0\end{bmatrix} \,\begin{bmatrix} 0&1\\1&0\end{bmatrix} =\begin{bmatrix} 0&0\\0&1\end{bmatrix}. $$

Related Question