SVDs of the matrix $\begin{bmatrix} 1 & 1 \\ 0 & 0\end{bmatrix}$

linear algebramatricesmatrix decompositionsvd

This is from Trefethen's Numerical Linear Algebra, lecture 4, exercise 1(d).

Determine the SVDs of the matrix
$$\begin{bmatrix} 1 & 1 \\ 0 & 0\end{bmatrix}$$


This matrix maps both $[1, 0]$, $[0, 1]$ to $[1, 0]$, and there is $1$ as a singular value. The corresponding singular vector is $[1, 0]$. Thus the SVD is $AV= U\Sigma$
$$\begin{bmatrix} 1 & 1 \\ 0 & 0\end{bmatrix}\begin{bmatrix} 1 & 0\\ 0 & 1 \end{bmatrix} = \begin{bmatrix} 1 & ? \\ 0 & ?\end{bmatrix}\begin{bmatrix} 1 & 0 \\ 0 & ?\end{bmatrix} $$

Since the given matrix $A$ doesn't have full rank, I'm having trouble filling in the empty entries so that the $U$ matrix is unitary.

Best Answer

Here's a procedure you can use to find the SVD of $A$. It's quite "quick and dirty" in the sense that I've just shown the procedure and not explained anything, but hopefully, it'll help as a guide for when you learn the general procedure from someone much more qualified than I :)

Let $A=\begin{bmatrix} 1 & 1 \\ 0 & 0\end{bmatrix}$ with SVD $A=U\Sigma V^T$.

First, find the eigenvalues of $A^TA$. So we have $$\begin{bmatrix} 1 & 0 \\ 1 & 0\end{bmatrix} \begin{bmatrix} 1 & 1 \\ 0 & 0\end{bmatrix}=\begin{bmatrix} 1 & 1 \\ 1 & 1\end{bmatrix}.$$ With eigenvalues $\lambda_1=2,\lambda_2=0$ both with algebraic multiplicity $1$. The singular values of $A$ are therefore $\sigma_1=\sqrt2$ and $\sigma_2=0$.

Next, find the eigenspaces corresponding to each eigenvalue. For $\lambda_1=2$, we have $$ E_{\lambda_1}=\ker\left(\begin{bmatrix} -1 & 1 \\ 1 & -1\end{bmatrix}\right)=\mathrm{span}\left\{\begin{bmatrix} 1 \\ 1\end{bmatrix}\right\}. $$ And for $\lambda_2=0$ we have $$ E_{\lambda_2}=\ker\left(\begin{bmatrix} 1 & 1 \\ 1 & 1\end{bmatrix}\right)=\mathrm{span}\left\{\begin{bmatrix} -1 \\ 1\end{bmatrix}\right\}. $$ Now take the union of the spanning sets for the eigenspaces and make the set orthonormal. The Gram–Schmidt process is the way to go. We obtain$$\left\{\begin{bmatrix} \frac{\sqrt2}{2} \\ \frac{\sqrt2}{2}\end{bmatrix},\begin{bmatrix} -\frac{\sqrt2}{2} \\ \frac{\sqrt2}{2}\end{bmatrix}\right\}.$$ Name these vectors $v_1$ and $v_2$, respectively. Then, define $$ u_1=\frac{1}{\sigma_1}Av_1=\begin{bmatrix} 1 \\ 0\end{bmatrix}. $$ We would like the define $u_2$ in the same way $u_2=\frac{1}{\sigma_2}Av_2$, but $\sigma_2=0$, so instead we set $u_2$ to be a matrix orthonormal to $u_1$. Clearly, $$ u_2=\begin{bmatrix} 0 \\ 1\end{bmatrix}. $$

Finally, we obtain following the matrices: $$ \begin{align} V&=\begin{bmatrix} v_1 & v_2\end{bmatrix}=\begin{bmatrix} \frac{\sqrt2}{2} & -\frac{\sqrt2}{2}\\ \frac{\sqrt2}{2} & \frac{\sqrt2}{2}\end{bmatrix}\\ U&=\begin{bmatrix} u_1 & u_2\end{bmatrix}=\begin{bmatrix} 1 & 0 \\ 0 & 1\end{bmatrix}\\ \Sigma&=\begin{bmatrix} \sigma_1 & 0 \\ 0 & \sigma_2\end{bmatrix}=\begin{bmatrix} \sqrt2 & 0 \\ 0 & 0\end{bmatrix}. \end{align} $$

Hope this helps!