Decompose Frobenius norm of the difference between 2 matrices

linear algebralinear-transformationsmatricesmatrix decompositionreal-analysis

Let $A\in\mathbb{R}^{n\times n}$ and $B = UDV^T$ where $U\in\mathbb{R}^{n\times r}$, $V\in\mathbb{R}^{n\times r}, D\in\mathbb{R}^{r\times r}$ and $U^TU = V^TV = I_r$. Define two sets:
$$S_1=\mathbb{R}^{r\times r}$$
$$S_2 = \{\text{All $r\times r$ diagonal matrices}\}\subset S_1$$
and
$$D_1=\arg\min_{M\in S_1}\|UMV^T – A\|_F^2$$
$$D_2=\arg\min_{M\in S_2}\|UMV^T – A\|_F^2$$
where $\|\|_F$ is the Frobenius norm.

My questions are:

  1. Is this identity true?
    \begin{align*}
    \|B-A\|_F^2&=\|B – UD_1V^T\|_F^2+\|UD_1V^T-A\|_F^2\\
    &=\|B – UD_2V^T\|_F^2+\|UD_2V^T-A\|_F^2
    \end{align*}

  2. Can we express $\|UD_1V^T-A\|_F^2$ and $\|UD_2V^T-A\|_F^2$ explicitly as functions of $U,V,A$?

Best Answer

  1. Note that the Frobenius norm $$ || X + Y ||_{F}^2 = tr\left( (X+Y)^T (X+Y) \right) = || X ||_{F}^2 + 2 tr\left( X^T Y \right) + || Y ||_{F}^2 $$ The minimization (defining $D_1$) yields that $$ tr\left( (UD_1 V^T - A)^T (U M V^T ) \right) = 0 $$ for any matrix $M$ in $S_1$. Next write $$ ‖𝐵−𝐴‖_F^2 = ||B - UD_1 V^T ||_F^2 + 2 tr\left( (UD_1 V^T - A)^T (B - UD_1 V^T ) \right) + || U D_1 V^T - A ||_F^2 $$ The middle term is $0$ because $B - UD_1 V^T = U( D - D_1 )V^T$.
  2. Write $$ || U D V^T - A ||_{F}^2 = || U ( D - U^T A V ) V^T ||_{F}^2 = || D - U^T A V ||_{F}^2 $$ So we get $D_1 = U^T A V$ and $D_2$ is the diagonal part of $U^T A V$.
Related Question