The norm $||A||=\max_{\langle u|u\rangle=1}|\langle u|A|u\rangle|$ for nondiagonalizable matrices

eigenvalues-eigenvectorslinear algebramatrix-normsquantum-information

In my reference Appendix 6: Proof of Lieb’s theorem, Page 645, Quantum Computation and Quantum Information by Nielsen and Chuang, it is given that

A matrix norm of $A$ is defined as $$||A||=\max_{\langle u|u\rangle=1}|\langle u|A|u\rangle|=\max_{u^Tu=1}|u^TAu|$$
If $A$ has eigenvalues $\lambda_i$ and defining $\lambda$ to be the maximum of the set $|\lambda_i|$ then

  1. $||A||\ge \lambda$
  2. When $A$ is hermitian, $||A||=\lambda$
  3. When $A=\begin{bmatrix}1&0 \\ 1&1\end{bmatrix}$, $||A||=3/2>1=\lambda$

My understanding of a similar matrix norm, called $L2$ norm is that $||A||_2=\max_{u^Tu=1}||Au||$ which is the largest singular value $\sigma_1$ of the matrix $A$, ie., square root of the largest eigen value of $A^TA$ and $\sigma_1^2=\max_{u^Tu=1}u^TSu$ with $S=A^TA$.

How do I show that $||A||\ge\lambda$?

For any fixed symmetric matrix $A\in\mathbb{R}^{n\times n}$, fixed rectangular matrix $B\in\mathbb{R}^{m\times n}$ and fixed vector $a\in\mathbb{R}^n$, we can define
\begin{align}
\frac{\partial}{\partial u}a^Tu=a\quad &\quad \frac{\partial}{\partial u}|u|^2=2u\\
\frac{\partial}{\partial u}(u^TAu)=2Au\quad &\quad \frac{\partial}{\partial u}|Bu|^2=2B^TBu\\
\end{align}
We need to find the critical points of the function $u^TAu$ subject to the constraint $|u|^2=u^Tu=1$.

ie., to find the critical points of the Lagrangian function, $L(u,\lambda)=u^TAu-\lambda(u^Tu-1)$
\begin{align}
\frac{\partial L}{\partial u}&=\frac{\partial}{\partial u}(u^TAu)-\lambda\frac{\partial}{\partial u}|u|^2\\
&=2Au-\lambda(2u)=0\\
\implies Au&=\lambda u\\
\frac{\partial L}{\partial\lambda}&=|u|^2-1=0\implies |u|^2=1
\end{align}
Therefore, $u,\lambda$ must be an eigenpair of $A$.

$\therefore$ the eigenvectors of $A$ are the critical points of $u^TAu$

$\therefore$ $\max_{u^Tu=1}u^TAu=\lambda_\max$, the largest eigenvalue of the matrix $A$.

So it looks like,
$||A||=\max_{\langle u|u\rangle=1}|\langle u|A|u\rangle|=\max_{u^Tu=1}|u^TAu|$ is the largest eigenvalue of the matrix $A$, whilst $ ||A||_2=\max_{u^Tu=1} ||Au|| $ is the largest singular value $\sigma_1$ of the matrix $A$. Am I looking at it correctly?

Given that the proof shows the critical values of $u^TAu$ are at the eigenvectors of $A$, how do I make sense of the case when $||A||>\lambda$?

For example, when $A=\begin{bmatrix}1&0 \\ 1&1\end{bmatrix}$, it is given that $||A||=3/2>1=\lambda$.

Best Answer

You can show the inequality $\|A\| \geq \lambda$ by taking for $u$ an unit eigenvector associated to $\lambda$. If $Au=\lambda u$, then $\langle u, Au\rangle = \lambda \|u\|^2.$

For 3., I suggest that you compute explicitely $\langle u, Au\rangle$ with $u=\pmatrix{u_1, u_2}$, $u_1^2+u_2^2=1$ and search for an explicit vector $u$ for which the equality holds.

Related Question