I would like to obtain a tight upper bound for the following matrix norm:
$$ \| I – \frac{x x^T}{\|x\|_2^2} \| $$
where $x$ is a column vector. (Clearly, the second term is a rank-1 normalized matrix)
I was able to apply the triangle inequality to obtain a lower bound, i.e., for two matrices $A,B$, we have: $ \| A – B \| \geq \|A\| – \|B\|$ but am not sure how to do the upper bound.
Best Answer
Call your matrix $M$, let $V$ be the vector space it acts on.
Write $V=A\oplus B$ where $A$ is the span of $x$ (it's one dimensional) and $B=A^\perp$ is the set of vectors perpendicular to $x$. If $v=a+b$ with $a\in A$ and $b\in B$. then $Mv = Mb = b$. Then we have $$ \frac {\|M(a+b)\|_2^2}{\|a+b\|_2^2} = \frac{\|b\|_2^2}{\|a\|_2^2+\|b\|_2^2} \le 1.$$
If $B$ is trivial (which happens only if $V$ is one dimensional) then $\|M\|=0$. Otherwise, the bound $\|M\|=1$ is achieved by taking $v=b$ for non-zero $b\in B$.