I want to show that $\|I_n\|_2=1$ but I'm not sure how should I calculate its largest singular value, am I supposed to use SDV decomposition each time I want to calculate a two-norm?
According to my book (numerical linear algebra by Gill), one norm (the maximum absolute column sum) and infinity norm (the maximum absolute row sum) for the following matrix would be $1$, right? What about its Frobenius norm?
Best Answer
Note that the identity matrix is a diagonal matrix with non-negative diagonal elements, hence its singular value decomposition is simply
$$I_n = I_n \cdot I_n \cdot I_n^\ast,$$
all its singular values are $1$, whence $\lVert I_n\rVert_2 = 1$.
Not necessarily. Note that an alternative way to define the two-norm is
$$\lVert A\rVert_2 = \sup_{\lVert x\rVert_2 = 1} \lVert Ax\rVert_2.$$
With the singular value decomposition
$$A = U\Sigma V^\ast,$$
we obtain
$$\begin{align} \sup_{\lVert x\rVert_2 = 1} \lVert Ax \rVert_2 &= \sup_{\lVert x\rVert_2 = 1} \lVert U\Sigma V^\ast x\rVert_2\\ &= \sup_{\lVert x\rVert_2 = 1} \lVert \Sigma V^\ast x\rVert_2\\ &= \sup_{\lVert y\rVert_2 = 1} \lVert \Sigma y\rVert_2, \end{align}$$
since $U$ and $V^\ast$ are norm-preserving, and it is not hard to see that $\sup_{\lvert y\rVert_2 = 1} \lVert \Sigma y\rVert_2$ is the largest singular value.
This supremum is in general not easy to find, but with that, we can see
$$\lVert A\rVert_2^2 = \sup_{\lVert x\rVert_2 = 1} \lVert Ax\rVert_2^2 = \sup_{\lVert x\rVert_2 = 1} \langle Ax, Ax\rangle = \sup_{\lVert x\rVert_2 = 1} \langle x, A^\ast Ax\rangle,$$
and $A^\ast A$ is a positive semidefinite hermitian matrix, thus $\lVert A\rVert_2^2$ is the largest eigenvalue of $A^\ast A$, which sometimes is easier to compute.
However, computing operator norms like $\lVert A\rVert_2$ is in general not a trivial task.
Regarding the Frobenius norm,
$$\lVert I_n \rVert_F = \sqrt{\sum_{i=1}^n \sum_{j=1}^n \delta_{ij}^2} = \sqrt{n},$$
where $\delta_{ij}$ is the Kronecker symbol, $\delta_{ij} = 1$ if $i = j$, and $\delta_{ij} = 0$ if $i\neq j$.