In many textbooks of pattern recognition I have seen the following statement:
If a matrix $A$ in the quadratic form $F(\text{x}) = \text{x}^{T}A\text{x}$ is positive definite, then it follows that the surfaces of constant $F(\text{x})$ are hyperellipsoids, with principal axes having lengths proportional to $\lambda_{k}^{-1/2}$.
$A$ is a $n \times n$ matrix, $\lambda_k$ is an eigenvalue of $A$, $\text{x}$ is a non-zero vector and $k = 1, …, n$.
My question is: "Why are the principal axes of hyperellipsoids proportional to the eigenvalues of $A$? Why are they proportional to $\lambda_{k}^{-1/2}$? And why are the surfaces of constant $F(\text{x})$ hyperellipsoids?"
Can someone give me a proof? Visualization perhaps? etc. Anything that would make me see that this indeed is the case 🙂
Hope my question is clear 🙂
Thank you for any help!
Best Answer
If $A \in \def\R{\mathbb R}\R^{n\times n}$ is positive definite, there is an orthogonal $Q \in O(n)$ (that is a matrix whose columns form an orthonormal basis of $\R^{n}$) such that $QAQ^t = \mathrm{diag}(\lambda_1, \ldots, \lambda_n) =: \Lambda$ with the $\lambda_k$ positive. For $c > 0$ and $x \in \R^n$ we have \begin{align*} F(x) &= c\\ \iff x^t Ax &= c \\ \iff x^t Q^t QAQ^t Qx &= c\\ \iff (Qx)^t \Lambda Qx &= c\\ \iff \sum_k \lambda_k (Qx)_k^2 &= c\\ \iff \sum_k \frac{(Qx)_k^2}{(c^{1/2}\lambda_k^{-1/2})^2} &= 1 \end{align*} So in the transformed coordinates $y = Qx$ given by $Q$ we see that the hypersurface $\{F = c\}$ is given by $$ \frac{y_1^2}{(c^{1/2}\lambda_1^{-1/2})^2} + \cdots + \frac{y_n^2}{(c^{1/2}\lambda_n^{-1/2})^2} = 1 $$ that is it forms a hyperellipsoid with principal axes in direction $Qe_k$ (where $e_k$ denotes the $k$-th standard unit vecotr) having lengths $c^{1/2}\lambda_k^{-1/2}$.