Consider a matrix $A \in \mathbb{R}^{m\times n}$ with singular values $\sigma_{i}$ after SVD.
What are the singular values of $AX$ when $X\in \mathbb{R}^{n\times r}$, and $X^{T}X = I$?
I think the singular values will change since the dimension of the matrix changes. However, I don't know how the singular values change.
Can anyone provide some ideas about this question?
Best Answer
The first (and largest) singular value of $A$ is characterized by $\max_{\|v\|=1} \|Av\|$. If $v_1$ is the first right singular vector, then the second singular value is given by $$\max_{\|v\|=1, v \perp v_1} \|Av\|.$$ And we can repeat this procedure. So taking the maximum value of $\|Av\|$ on the set of unit vectors in $\mathrm{span}\{v_1,\ldots,v_k\}^\perp$ (the orthogonal complement of the space spanned by the first $k$ right-singular vectors), will give us the next singular value.
By applying this procedure to the matrix $AX$, we get that the largest singular values of $AX$ is smaller than that one of $A$. Since the image of $X$ is not the whole of $\mathbb{R}^n$, the maximum $\|AXu\|$ will be smaller than the first singular value of $A$. With a similar argument, the smalles singular value of $AX$ must be bigger then the smallest singular value of $A$.
A particular intersting case is when $X = [v_1\;v_2 \;\cdots\; v_k]$. In this case, the singular values of $AX$ are the first $r$ singular values of $A$.