Consider an ellipsoid $\{x| x^TAx = 1\}$.
Let $A$ be a real symmetric matrix, and consider the eigen-decomposition
$A=P\Lambda P^{-1} =P\Lambda P^T$,
where the matrix $P$ is orthogonal because $A$ is symmetric, and the diagonal matrix $\Lambda$ contains the eigenvalues of $A$.
Then the ellipsoid can be rewritten as:
$$x^TP\Lambda P^Tx = (P^Tx)^T\Lambda(P^Tx) = y^T\Lambda y = 1$$
Since the matrix is diagonal, the quadratic form gives:
$$y^T\Lambda y =\lambda_1y_1^2+…+\lambda_ny_n^2 = 1$$
This is clearly the equation of ellipsoid.
My question is, why are the axes of this ellipsoid the eigenvectors of $A$?
Best Answer
The axes of this ellipsoid are (the multiples of) $e_1=(1,0,0,\ldots,0)$, $e_2=(0,1,0,\ldots,0)$, …, $e_n=(0,0,0,\ldots,1)$ (and the corresponding eigenvalues are $\lambda_1,\lambda_2,\ldots,\lambda_n$). But $\Lambda=P^{-1}AP$ and $P$ is a change-of-bases matrix. So, the columns of $P$ are eigenvectors of $A$ (and, again, the corresponding eigenvalues are $\lambda_1,\lambda_2,\ldots,\lambda_n$). This was just a change of basis, so, geometrically, the columns of $P$ are the still the vectors $e_1,\ldots,e_n$.