Every (real) $m\times n$ matrix $A$ of rank $r$ has an SVD
$$
A = U\Sigma V^T
$$
Now, I have read about the following properties:
- $\text{Image}(A) = \text{span}\{u_1,\dots,u_r\}$
- $\text{Null space}(A) = \text{span}\{v_{r+1},\dots,v_n\}$
Maybe I am lacking some knowledge from my linear algebra courses, but how can these properties be proven?
Best Answer
Image(A) means column space of A. I will assume $m\geq n$ and rank(A) = r without loss of generality. To see these identity, distinguish compact and full svd decomposition: \begin{align} A = U_1\Sigma_1 V_1^T = \underbrace{[U_1,\ U_2]}_U\underbrace{\begin{bmatrix} \Sigma_1 & 0\\0 & 0\end{bmatrix}}_\Sigma\underbrace{\begin{bmatrix} V_1^T \\ V_2^T\end{bmatrix} }_V \end{align} where $U \in R^{m\times m}$, $\Sigma = diag(\sigma_1,\ldots,\sigma_r)\in R^{r\times r}$, and $V \in R^{n\times n}$. Matrix U constitutes an orthonormal basis for $R^{m\times m}$ with $$U^TU=UU^T = I_m.$$ In the same way, matrix V constitutes an orthonormal basis for $R^{n\times n}$ with $$V^TV=VV^T=I_n.$$ On the other hand, by compact SVD, you get only orthonormal basis for the range and domain space of matrix A. So, you do not have the property $$U_iU^T_i = I_m \ or \ V_iV^T_i = I_n, \ i = 1,2. $$ Knowing this distinction, come back to the your question. Consider following matrix transformation for $\forall x\in R^n$ \begin{align} Ax &= \sum_i^r \sigma_i(v_i^Tx)u_i \\&= U_1\Sigma_1V_1^Tx = U_1z\\ &=\sum_i^rz_iu_i \end{align} where $z_i = \sigma_i(v_i^Tx)$. So, Ax is just weighted linear combination of vectors $\{u_1,\ldots,u_r\}$ which equivalent to say $\mathcal{R}(A) = span\{u_1,\ldots,u_r\}$. On the other hand, null space is the vectors that are mapped to zero, i.e. if $x\in \mathcal{N}(A)$, then $Ax = 0$. Since the set $\{u_1,\ldots,u_r\}$ is orthonormal, \begin{equation} Ax = U_1z = 0 \end{equation} can be possible only if z is zero vector. \begin{align} z = \Sigma_1V_1Tx = 0 \leftrightarrow V_1^Tx \end{align} $\Sigma$ is just nonzero diagonal matrix, we can ignore it. So, vector z is $0$ if and only if vector x is perpendicular to $span\{v_1,\ldots,v_r\}$ which means that $$x\in span\{v_{r+1},\ldots,v_n\} = \mathcal{R}(V_2)\rightarrow Ax = 0$$