I have read that we define the "2-norm" of a matrix as
$$\max_i \,{|\sigma_i|},$$
which I have also heard called the "operator norm" (here $\sigma_i$ are the singular values).
Also we have the norms
$$\|A\| = \left( \sum_{i,j}|a_{ij}|^q \right)^{1/q}$$
for every $q\geq 1$. Do we refer to these as $\|A\|_q$? (For $q=2$, I have heard this referred to as the "Frobenius norm".) If we do refer to them as $\|A\|_q$, then how can we reconcile the two meanings of the term "two-norm"?
Subquestion: How can we bound the values of $\|A\|$ for $q=1$ and $q=2$ in terms of each other?
Best Answer
I think I understand your question - typically $\|A\|_2$ has two definitions $$ \|A\|_2 = \sqrt{\text{largest eigen value of } A^{\ast}A} $$ and $$ \|A\|^{\prime}_2 = \sup_{\|x\|_2 = 1}\|Ax\|_2 $$ Note that $B = A^{\ast}A$ is a symmetric, positive matrix ($\langle Bx,x \rangle \geq 0$), and hence it can be diagonalized, and its eigen values are non-negative. Write them as $$ \lambda_1 \geq \lambda_2 \geq \ldots \geq \lambda_n \geq 0 $$ and consider an orthonormal basis $\{u_1, u_2, \ldots, u_n\}$ such that $$ Bu_i = \lambda_i u_i $$ For any $x \in \mathbb{R}^n$, write $x = \sum \alpha_i u_i$, then $$ \|x\|_2 = 1 \Leftrightarrow \sum_{i=1}^n \alpha_i^2 = 1 $$ So consider $$ \|Ax\|^2_2 = \langle Ax,Ax\rangle = \langle A^{\ast}Ax,x\rangle = \sum_{i=1}^n \lambda_i \alpha_i^2 $$ Hence, it follows that $$ \|Ax\|^2_2 \leq \lambda_1 $$ and hence $\|A\|^{\prime}_2 \leq \sqrt{\lambda_1} = \|A\|_2$
The other inequality is obvious - can you see that?