Problem
This looks like a stupid question, but the following is not straightforward to me
$$
\Vert \mathbf{W}\Vert_p \geq \Vert \mathbf{\tilde{W}}\Vert_p, \Vert \mathbf{W}\Vert_F \geq \Vert \mathbf{\tilde{W}}\Vert_F
$$
where $\mathbf{W}=\sum_{i=1}^r s_i \mathbf{u}_i\mathbf{v}_i^T$ ($r$ is number of non-zero singular values) and $\mathbf{\tilde{W}}= s_1 \mathbf{u}_1\mathbf{v}_1^T$
What I Have Done
I tried to write $\mathbf{W}$ in the following
$$
\mathbf{W}=\mathbf{\tilde{W}}+\sum_{i=2}^r s_i \mathbf{u}_i\mathbf{v}_i^T
$$
But something like
$$\Vert \mathbf{\tilde{W}}+\sum_{i=2}^r s_i \mathbf{u}_i\mathbf{v}_i^T\Vert_p \geq \Vert \mathbf{\tilde{W}}\Vert_p$$ does not seem to hold.
Could someone help me, thank you in advance.
Best Answer
If $W = U \Sigma V^T$ is the singular value decomposition of $W$, then $$\|W\|_F^2 = \text{Tr}(W W^T) = \text{Tr}(U \Sigma^2 U^T) = \text{Tr}(\Sigma^2) = \sum_i s_i^2$$ while $\|\tilde{W}\|_F^2 = s_1^2$.
EDIT: It's not true for the $p$-norm. For example, in dimension $2$ take $p=1$ with ${\bf u}_1 = {\bf v}_1 = (3/5, 4/5)^T$, ${\bf u}_2 = {\bf v}_2 = (4/5, -3/5)^T$, $s_1 = s_2 = 1$. Then ${\bf W} = I$ has norm $1$, but $$ \widetilde{\bf W} = \pmatrix{9/25 & 12/25\cr 12/25 & 16/25\cr}$$ has norm $28/25$, with $$\widetilde{\bf W} \pmatrix{0\cr 1\cr} = \pmatrix{12/25\cr 16/25}$$