So, I was reading a paper, and a step in the proof stumbled me.
$$
U \in \mathbb{R}^{m \times k} (GIVEN: Orthogonal \ matrix), M \in \mathbb{R}^{m \times n}, M = U^*\Sigma^*{V^*}^T (rank-k \ SVD)
$$
$$
||(I – UU^T)U^*\Sigma^*{V^*}^T||^2_2 = || U_{perp}U^*\Sigma^*||^2_2
$$
where $U_{perp}$ is the basis of the orthogonal complement of the subspace spanned by the columns of U.
I have toiled hard to understand this, but I cannot see why this should hold.
Reference: https://arxiv.org/pdf/1212.0467v1.pdf.
Lemma: C.1 (Page – 34)
Thanks.
Best Answer
It may help to note that $UU^T$ is the projection onto the span of the columns of $U$, and that $I-UU^T$ is the projection onto the orthogonal complement. We can therefore write $$ I-UU^T= U_{perp}U_{perp}^T $$ I think that should help. Not also that $\|A\|=\|AU\|=\|VA\|$ for any orthogonal matrices $U$ and $V$.