Related to my work is the concept of Singular Value Decomposition (SVD). Namely, given some matrix $B\in\mathbb{R}^{n\times m}$, $n\geq m$, SVD can be written as
$$B=U\Sigma V^T,$$
where $U\in\mathbb{R}^{n\times n}$ ($n$ left-singular vectors), $\Sigma \in\mathbb{R}^{n\times m}$ (matrix with $m$ singular values on its diagonal), and $V\in\mathbb{R}^{m\times m}$ ($m$ right-singular vectors). So, how I see it, given that there are $m$ singular values, only right-singular vectors and $m$ first left-singular vectors have the asociated singular value? To which singular value do the rest of left-singular vectors correspond to?
In case one complements $\Sigma$ to contain zero singular values such that $\Sigma\in\mathbb{R}^{n\times n}$, then $V^T$ needs also to be complemented to be a $V^T\in\mathbb{R}^{n\times m}$ matrix. But, that would imply that a vector of all zeros is also a right-singular vector. Could someone clarify this?
Furthermore, suppose that the rank of matrix $B$ is $c$, $c<m\leq n$. Does this imply that $B$ has only $d$ left- and right-singular vectors?
Best Answer
The left singular vectors of $B$ are eigenvectors of $BB^T$, and the right singular vectors of $B$ are eigenvectors of $B^TB$, so you can easily see why there are $n$ left singular vectors, and $m$ right singular vectors.
If $n \ge m$, then there are only $m$ singular values, which are the square root of eigenvalues of $BB^T$. If $m \ge n$, then there are only $n$ singular values, which are the square root of eigenvalues of $B^TB$. So the rest of left singular vectors do not correspond to singular values, there are the eigenvectors for the 0 eigenvalue of $BB^T$.
We always have $\Sigma \in\mathbb{R}^{n\times m}$, not $\Sigma \in\mathbb{R}^{n\times n}$ if $n > m$. Zero vectors are not singular vectors.
There are always $n$ left singular vectors, and $m$ right singular vectors.