[Math] Why is the condition number of a matrix given by these eigenvalues

condition numberlinear algebramatricesnumerical linear algebra

${\bf A}$ is $n \times n$ matrix. I want to know why the condition number of ${\bf A}$ is given by:

$$\frac{\rm{max}(\lambda_i)}{\rm{min}(\lambda_j)},$$

where $\lambda$ are eigenvalues of ${\bf A}$. Would you please give me a proof?

This equation come from the page 80 in "Deep Learning" written by Ian Goodfellow.

Best Answer

In the book, the condition number refers to the matrix $A \in \mathbb{R}^{n \times n}$, not the function as you stated in the question.

The condition number of a matrix $A$ is defined as

$$ \kappa(A) = \|A\|_2\|A^{-1}\|_2,$$

where $\| \cdot \|_2$ is spectral norm of a matrix. It is known that the spectral norm of a matrix equals its maximum singular value

$$ \|A\|_2 = \sigma_{max}(A) $$

and that the maximum singular value of $A^{-1}$ equals 1 over the minimum singular value of $A$

$$ \sigma_{max}(A^{-1}) = 1 / \sigma_{min}(A).$$

Thus,

$$ \kappa(A) = \sigma_{max}(A) / \sigma_{min}(A).$$

If the matrix $A$ is normal (which means $A$ can be decomposed as $A=Q \Lambda Q^T$ where $Q$ is an orthogonal matrix and $\Lambda$ is a diagonal matrix whose entries are the eigenvalues of $A$), and using the fact that $\sigma_i(A) = \sqrt{\lambda_i(A^TA)}$, we have

$$ \sigma_{max}(A) = \sqrt{\lambda_{max}(A^TA)} = \sqrt{\lambda_{max}((Q\Lambda Q^T)^TQ\Lambda Q^T)} = \sqrt{\lambda_{max}(Q\Lambda^2Q^T)} = \sqrt{\lambda_{max}(A)^2} = |\lambda_{max}(A)|, $$

and then we have

$$ \kappa(A) = |\lambda_{max}(A)|/|\lambda_{min}(A)|.$$

However, in the book, I do not see where the author mentions that the matrix $A$ is normal.