Physicist here trying to understand the equivalent definitions of the operator norm. Please be gentle.
I think I understand the definition of the operator norm
$$
\|A\|_{\rm op} = \inf \{c : \|Av\| \leq c\|v\| \textrm{ for all } v \in V \}
$$
but I don't see how the following equivalent statements are equal
$$
a = \sup\{\|Av\| : v \in V \textrm{ with } \|v\| \leq 1 \}
$$
$$
b = \sup\{\|Av\| : v \in V \textrm{ with } \|v\| = 1 \}.
$$
In the answers to this question, someone says "notice that $b \leq a$", whilst someone else claims that "$a \geq b$ is easily seen". In each case there is no elaboration on why those statements might be true.
Would be great if someone could please walk me through this in baby steps.
To give you an idea of my level of knowledge: it's taking me way too long to work out what tags to use for this question.
Best Answer
Let's look first at why we also have $a \le b$. For that take $v \in V$ with $||v|| <1$. Then we have $$||Av|| = || \ ||v|| A \frac{v}{||v||} \ || = ||v|| \cdot ||A \frac{v}{||v||} || < ||A \frac{v}{||v||} ||$$ where $\frac{v}{||v||}$ has norm $1$. Therefore the supremum $a$ can never be reached in a point $v \in V$ with $||v|| <1$. With the fact $a \ge b$ as said in the comments we get $a=b$.
Now: why is $a$ equal to the operator norm? We look at the following $$||Av|| \le c ||v|| \Leftrightarrow \frac{||Av||}{||v||} \le c \Leftrightarrow ||A \frac{v}{||v||} || \le c$$ So when we look for the smallest $c$ satisfying this inequality for every $v$ we can alternatively look at the supremum $$\sup_{ v \in V} \{||A \frac{v}{||v||} \}$$ which is the same as $$a= \sup_{||v||=1} \{||Av||\}$$ because the following two sets are equal: $$\{ \frac{v}{||v||}: v \in V \} = \{v \in V: ||v||=1\}$$ (You can write every vector of norm 1 as any vector divided by its norm)
And all in all this gives us $||A||_{op} = a=b$.