Could someone explain the second equality in the definition of a induced matrix norm to me?
Let $\| \cdot \|$ be a norm for $\cdot \in \mathbb{R}^n$,
then the induced matrixnorm for $A\in \mathbb{R}^{n\times n}$ is given by:$$\|A\| = \sup_{x\not = 0} \frac{\|Ax\|}{\|x\|} \color{red}{\stackrel{?}{=}} \max_{\|x\|= 1} \|Ax\|$$
Problem:
Why does $\color{red}{=}$ hold?
I know $\|Ax\|$ is continous for all $x\in \mathbb{R}^n$ and
$\{x \in \mathbb{R}^n :\|x\| = 1 \}$ is compact which implies $\dfrac{\|Ax\|}{\|x\|}$
to reach a maximum according to Weierstrass.
But why is this maximum also the maximum for all $x$? (even those with norm bigger or less then 1?)
Best Answer
I guess it is clear that we have $$ \sup_{x\neq 0} \frac{\Vert Ax\Vert}{\Vert x\Vert} \geq \max_{\Vert x\Vert = 1} \Vert A x\Vert $$ because $A:=\{ x \in \mathbb{R}^n \mid \Vert x \Vert = 1 \} \subseteq \{ x\in\mathbb{R}^n \mid x \neq 0\}=:B$.
The other way around, assume that $y\in B$. Then we have $z:= \frac{y}{\Vert y\Vert} \in A$ and thus $$ \frac{\Vert Ay\Vert}{\Vert y\Vert}= \left\Vert A\frac{y}{\Vert y\Vert}\right\Vert= \Vert Az\Vert \leq \max_{x\in A}\Vert Ax\Vert = \max_{\Vert x\Vert = 1} \Vert A x\Vert. $$ Because $y\in B$ was arbitrary this inequality holds for all $y\in B$. In particular, we get $$ \sup_{y \neq 0} \frac{\Vert Ay\Vert}{\Vert y\Vert} = \sup_{y\in B} \frac{\Vert Ay\Vert}{\Vert y\Vert} \leq \max_{\Vert x\Vert = 1}\Vert A x\Vert\text{.} $$ Both inequalities together give the desired equality.