[Math] Full and reduced SVD of a 3×3 matrix.

linear algebrasvd

I currently studying for an exam, and I'm currently working my way through some old exam problems and I'm currently at the following.

First, we have a matrix

$A=
\begin{bmatrix}
2&0&0\\2&1&0\\0&-2&0
\end{bmatrix}$

  • a. Determine the singular values of the matrix A.
  • b. Write down the reduced SVD-decomposition of A.
  • c. Determine the full SVD-decomposition of A.
  • d. Let $C=A^*A$ and $D=AA^*$. Determine whether these are positive semidefinite.
  • e. Are they positive definite?

My answers are,
For readers ease I write down the formulars.

$A=U\Sigma V^T$, where $U$ and $V$ are orthogonal and $\Sigma$ is a diagonal matrix.

$A^TA=V\Sigma^T \Sigma V^T$

$AV=U\Sigma$.

a. First i find the eigenvalues of $A^TA$, the root these values will be my singular values for A.

$\det(A^TA-\lambda I)=\begin{vmatrix}
8-\lambda&2&0\\2&5-\lambda&0\\0&0&-\lambda
\end{vmatrix}=\lambda(\lambda-4)(\lambda-9)$

Thus, $\lambda_1=9, \lambda_2=4$ and $\lambda_3=0$.

So the singular values is 3,2 and 0. I ordered the singular values such that $\Sigma$ has increasing values along the diagonal to simplify notation. $\Sigma=\begin{bmatrix}3&0&0\\0&2&0\\0&0&0\end{bmatrix}$

b. I now find $V=(e_1,e_2,e_3)$, where $e_j,j=1,2,3$ is the unit vector corresponding to the the $j'th$ eigenvalue. I compute these by applying the Gram-Schmidt procedure to the arbitrary list of eigenvectors for $A^TA$.

$V=\begin{bmatrix}
\frac{2}{\sqrt5}&\frac{1}{\sqrt5}&0\\
\frac{1}{\sqrt5}&-\frac{2}{\sqrt5}&0\\
0&0&1
\end{bmatrix}$

Now I only need to find U, because $V=V^T$.
I find U by using the formular $AV=U\Sigma$.

$AV=\begin{bmatrix}
2&0&0\\2&1&0\\0&-2&0
\end{bmatrix}\begin{bmatrix}
\frac{2}{\sqrt5}&\frac{1}{\sqrt5}&0\\
\frac{1}{\sqrt5}&-\frac{2}{\sqrt5}&0\\
0&0&1
\end{bmatrix}=\begin{bmatrix}
\frac{4}{\sqrt5}&\frac{2}{\sqrt5}&0\\
\sqrt5&0&0\\
-\frac{2}{\sqrt5}&\frac{4}{\sqrt5}&0
\end{bmatrix}$

$U\Sigma=\begin{bmatrix}
\frac{4}{\sqrt5}&\frac{2}{\sqrt5}&0\\
\sqrt5&0&0\\
-\frac{2}{\sqrt5}&\frac{4}{\sqrt5}&0
\end{bmatrix}
=
\begin{bmatrix}
\frac{4}{3\sqrt5}&\frac{1}{\sqrt5}&0\\
\frac{\sqrt5}{3}&0&0\\
-\frac{2}{3\sqrt5}&\frac{2}{\sqrt5}&0
\end{bmatrix}
\begin{bmatrix}3&0&0\\0&2&0\\0&0&0\end{bmatrix}$

Now I have all three components of the SVD-decomposition such that

$A=U\Sigma V^T=\begin{bmatrix}
\frac{4}{3\sqrt5}&\frac{1}{\sqrt5}&0\\
\frac{\sqrt5}{3}&0&0\\
-\frac{2}{3\sqrt5}&\frac{2}{\sqrt5}&0
\end{bmatrix}\begin{bmatrix}3&0&0\\0&2&0\\0&0&0\end{bmatrix}\begin{bmatrix}
\frac{2}{\sqrt5}&\frac{1}{\sqrt5}&0\\
\frac{1}{\sqrt5}&-\frac{2}{\sqrt5}&0\\
0&0&1
\end{bmatrix}$.

I believe that this answers both b. and c. because this is the reduced SVD and it's regarding a square matrix, so it's already a full SVD?

d. and e. First I calculate the matrices and then find the determinants of the upper left principals of the matrix, if they are all non-negative numbers, they will be positive semidefinite, if the determinants are strictly positive, they will be positive definite.

Knowing that $A$ is a real matrix, thus the adjoint of it is just its transpose, such that,

$C=\begin{bmatrix}
2&0&0\\2&1&0\\0&-2&0
\end{bmatrix}\begin{bmatrix}
2&2&0\\0&1&-2\\0&0&0
\end{bmatrix}=\begin{bmatrix}
4&4&0\\4&5&-2\\0&-2&4
\end{bmatrix}$

The determinants of the upper left principals will be, $4,4$ and $0$, so $C$ is positive semidefinite.

$D=\begin{bmatrix}
2&2&0\\0&1&-2\\0&0&0
\end{bmatrix}\begin{bmatrix}
2&0&0\\2&1&0\\0&-2&0
\end{bmatrix}=\begin{bmatrix}
8&2&0\\2&5&0\\0&0&0
\end{bmatrix}$

Here the determinants of the upper left principals is $8, 36$ and $0$, so this matrix is also positive semidefinite.

This concludes the problem.

I'm not totally sure about my answers, so I hope that I get some tips, tricks and corrections.
I'm really iffy about the argument of that the reduced SVD is the same as the full SVD for a square matrix.

Best regards Jens

Best Answer

I think you have the right general approach to computing the SVD. A couple notes:

  • Once you find the eigenvectors of $A^*A$, you know that eigenvectors will be orthogonal by the spectral theorem if their eigenvalues are distinct. So you only need to normalize them rather than do the entire Gram-Schmidt.
  • if you have a zero singular value is that you cannot compute $\sigma u = Av$ does not mean $u = Av / \sigma$. In your example, this lead to your $U$ missing a column (and therefore not being unitary). You can fix this by replacing it with a unit vector orthogonal to the other two columns.

The definition of reduced SVD varies some. If $A$ is $m\times n$ with $(m\geq n)$, the reduced SVD generally means the terms are factors liked $(m\times n), (n\times n), (n\times n)$. This is what you did, and so if $A$ is square, the reduced SVD would be the same as the regular SVD.

On the other hand, reduced SVD could mean $\Sigma$ is square and of size equal to the rank of $A$. In your case, this would mean dropping the last column of $U$, the last rows and columsn of $A$ and the last row of $V^*$.

All of these decompositions are equivalent since you are just dropping entries which end up multiplied by zero, but you should check which one you're using in the class.

For the positive definite questions, there are many equivalent definitions which might be faster to use. However, I'm not sure if you have seen them. Some useful ones are a hermitian matrix $M$ is positive definite if

  • all eigenvalues are positive
  • it can be factored $M = CC^T$ for some $C$ full rank.
  • for all $x \neq 0$, $x^TMx > 0$
Related Question