In my textbook there are few facts left without any sign of a proof, which really bugs me, and I was thinking maybe someone can help me:
- $A\in \mathbb{R}^{m,n} \Rightarrow \ \|A\|_2 = \sqrt{\rho(A^TA)}$ where $\rho(A^TA)$ denotes spectral radius of a matrix $A^TA$ i.e.
maximum absolute value of an eigenvalue of $A^TA$ (in this case all
eigenvalues are real and non-negative)- If $A\in\mathbb{R}^{n,n}$ and $A^T=A$ then $\|A\|_2=\rho(A)$
- $\|A\|_2\le\sqrt{\|A^TA\|_{\infty}}$
I suppose these are well known facts, yet I couldn't find their proofs on the Internet. Eigenvalues are very new for me, maybe that is why I couldn't (and why I am not able to prove it alone, I hope they are not too complicated, I prefer simple proofs). In first also fact that eigenvalues of $A^TA$ are real and non-negative is not easy for me.
Additionally I have several questions:
- I suppose that (yet I don't know a lot about it) $\|A\|_2=\max_{\vec{x}\neq\vec{0}} \frac{\|A\vec{x}\|_2}{\|\vec{x}\|_2}$ is not easy to compute. Is this second definition of $\|A\|_2$ (with spectral radius) helpful? I mean easier to compute? Can one easy and quite fast find eigenvalues, at least numerically?
- Is this inequality in the third point very good, that is $\|A\|_2$ is close to given upper bound?
Best Answer
This was asked here already many times: $$ \|A\|_2^2 = \max_{x\neq 0}\frac{\|Ax\|_2^2}{\|x\|_2^2}=\max_{x\neq 0}\frac{x^TA^TAx}{x^Tx}=\lambda_{\max}(A^TA)=\rho(A^TA). $$ The next to last equality is due to the Courant-Fischer theorem.
If $A$ is symmetric, then $\rho(A^TA)=\rho(A^2)=\rho(A)^2$. (Notice that symmetric matrices have real eigenvalues and if a $\lambda$ is an eigenvalue of a symmetric $A$, then $\lambda^2$ is a nonnegative eigenvalue of $A^2$.)
For any (square) $B$ and any matrix norm $\|\cdot\|$ subordinate with respect to some vector norm, we have $\rho(B)\leq\|B\|$. Plug that into the point 1) with the special choice $\|\cdot\|=\|\cdot\|_{\infty}$ to get $\|A\|_2=\sqrt{\rho(A^TA)}\leq\sqrt{\|A^TA\|_{\infty}}$.
Additional 1.
The largest eigenvalue of a matrix is usually quite easy to compute (or estimate) unlike the full spectrum or small eigenvalues but that might (and probably does) depend on the application. There is quite a selection of various numerical methods for computing eigenvalues: the full spectrum or just a part of it. Note that all such algorithms are iterative and may perform well or poorly depending on the matrix.
Additional 2.
IMHO it does not need to be the case generally (sometimes the bound can be tight, sometimes not). However, unlike $\rho(A^TA)$, the $\infty$-norm is almost trivial to compute.