Can someone point me to a paper, or show here, why symmetric matrices have orthogonal eigenvectors? In particular, I'd like to see proof that for a symmetric matrix $A$ there exists decomposition $A = Q\Lambda Q^{-1} = Q\Lambda Q^{T}$ where $\Lambda$ is diagonal.
Linear Algebra – Orthogonality of Eigenvectors of Real Symmetric Matrices
eigenvalues-eigenvectorslinear algebramatricesreference-requestsymmetric matrices
Related Solutions
Provided that the Schur decomposition is an allowed tool:
Using the Schur decomposition, we have that there exists an orthogonal $Q$ and an upper triangular $R$ such that $A=QRQ^T$. Since $A$ is symmetric, $Q^TAQ=R$ is symmetric as well. Therefore $R$ is symmetric. A symmetric triangular matrix is necessarily diagonal.
There is also a neat theory behind tridiagonal matrices, which can help:
It is easy to show that for any real $A$ there is an orthogonal matrix $Q$ such that $Q^TAQ=H$, where $H$ is upper Hessenberg. If $A$ is symmetric it then follows that $H$ is symmetric as well and hence tridiagonal. Now if the tridiagonal matrix $H$ is unreduced (none of the upper and lower diagonal entries are zero), then the eigenvalues of $H$ (and therefore of $A$) are distinct. Equivalently, if $A$ has repeated eigenvalues then $H$ is reduced (some upper and lower (symmetrically) diagonal entries are zero and hence $H$ is a block diagonal matrix). Consequently, one must have that each repeated eigenvalue must be in different unreduced diagonal blocks of $H$. In each of these blocks we must find at least one eigenvector (actually, only one) and it is almost trivial to show that these eigenvectors must be linearly independent (therefore, we can orthogonalize them).
A symmetric matrix with real entries is orthogonally diagonalizable.
In $\Bbb{R}^2$, $A$ is orthogonally diagonalizable means that $A$ has two orthonormal eigenvectors $u_1$ and $u_2$ with eigenvalue $\lambda_1$ and $\lambda_2$ respectively.
Suppose $x=c_1 u_1+c_2 u_2$, then $Ax=\lambda_1 c_1 u_1+\lambda_2 c_2 u_2$.
If $\lambda_1>0$ and $\lambda_2>0$. then $Ax$ can be gotten by stretching the plane (like a flat balloon) along the direction $u_1$ with $\lambda_1$ scale and along the direction $u_2$ with $\lambda_2$ scale.
The another equivalent aspect is as you said. Since $A=PDP^T$, where $P$ is orthogonal (stand for the rotation), transforming under $A$ is rotation, then scaling and revesing the rotation. But as Ian's comment, if there is a negative eigenvalue, we should choose the opposite direction when we were scaling. For example, $$A=\begin{pmatrix} 5 & -2\\ -2& 8\\ \end{pmatrix} =PDP^T= \begin{pmatrix} \frac{2}{\sqrt{5}} & \frac{-1}{\sqrt{5}}\\ \frac{1}{\sqrt{5}} & \frac{2}{\sqrt{5}}\\ \end{pmatrix} \begin{pmatrix} 4 & 0 \\ 0 & 9 \\ \end{pmatrix} P^T.$$ $\theta\approx 26.6^{\circ}$.
Best Answer
For any real matrix $A$ and any vectors $\mathbf{x}$ and $\mathbf{y}$, we have $$\langle A\mathbf{x},\mathbf{y}\rangle = \langle\mathbf{x},A^T\mathbf{y}\rangle.$$ Now assume that $A$ is symmetric, and $\mathbf{x}$ and $\mathbf{y}$ are eigenvectors of $A$ corresponding to distinct eigenvalues $\lambda$ and $\mu$. Then $$\lambda\langle\mathbf{x},\mathbf{y}\rangle = \langle\lambda\mathbf{x},\mathbf{y}\rangle = \langle A\mathbf{x},\mathbf{y}\rangle = \langle\mathbf{x},A^T\mathbf{y}\rangle = \langle\mathbf{x},A\mathbf{y}\rangle = \langle\mathbf{x},\mu\mathbf{y}\rangle = \mu\langle\mathbf{x},\mathbf{y}\rangle.$$ Therefore, $(\lambda-\mu)\langle\mathbf{x},\mathbf{y}\rangle = 0$. Since $\lambda-\mu\neq 0$, then $\langle\mathbf{x},\mathbf{y}\rangle = 0$, i.e., $\mathbf{x}\perp\mathbf{y}$.
Now find an orthonormal basis for each eigenspace; since the eigenspaces are mutually orthogonal, these vectors together give an orthonormal subset of $\mathbb{R}^n$. Finally, since symmetric matrices are diagonalizable, this set will be a basis (just count dimensions). The result you want now follows.