I'm at a loss of how to show this. I know that this is implied by the Spectral Theorem but not sure exactly how to show this in a simple, straightforward proof.
I've tried laying down what I know about symmetric matrices and diagonalizable ones but I'm not too sure what to use when. My guess is I'd have to reason with regards to the eigenvalues but again I'm not sure.
A matrix is symmetric if $A = A^T$. A matrix $A$ is symmetric if it can be expressed in the form $A = QDQ^{T}$. A square matrix $A$ is called diagonalizable if $\exists$ invertible P such that $P^{-1}AP$ is a diagonal matrix.
Would really appreciate any help.
Best Answer
For any symmetric matrix $A \in \mathbb{R}^{n \times n}$, consider the objective function,
$$ \begin{equation} \max_x x^T A x \\ \text{Given } \:x^Tx = 1 \end{equation} $$
The solution (In case of multiple solutions, consider any one) to the above objective function say $x_1$ is such that, $$Ax_1 = \lambda_1 x_1$$
(You could show this using Lagrangian function)
Now consider the next objective function,
$$ \begin{equation} \max_x x^T A x \\ \text{Given } \:x^Tx = 1 \text{ and } x^Tx_1 = 0 \end{equation} $$
The solution (In case of multiple solutions, consider any one) to the above objective function say $x_2$ is such that, $$Ax_2 = \lambda_2 x_2$$
(Again you could show this using Lagrangian function)
Continuing this procedure we could find the full set of eigen vectors.
The objective function at any iteration $i$ will be,
$$ \begin{equation} \max_x x^T A x \\ \text{Given } \:x^Tx = 1 \text{ and } x^Tx_1 = x^Tx_2 = \cdots x^Tx_{i-1} = 0 \end{equation} $$
The above objective function will always have a solution as long as the constraint set (which is actually a closed and bounded subset of $R^n$) is not an empty set, i.e., $$\{x:x^Tx = 1 \text{ and } x^Tx_1 = x^Tx_2 = \cdots x^Tx_{i-1}\} \neq \phi$$
The constraint set will be empty only when $x_1, x_2, \dots x_{i-1}$ span the $R^n$, which means they form a full set of orthonormal basic for $R^n$ which are actually eigen vectors of $A$.
Once you have $n$ independent (actually orthonormal) vectors, it is straightforward to show that $A$ is diagonalisable.