I am reading "Seminar of Linear Algebra" by Kenichi Kanatani.
In this book, there is the following problem:
Show that a symmetric and idempotent matrix $P$ is the orthogonal projection matrix onto some subspace.
The author's answer is here:
As is well known, an $n \times n$ symmetric matrix $P$ has $n$ real eigenvalues $\lambda_1, \dots, \lambda_n$, the corresponding eigenvectors $u_1, \dots, u_n$ being an orthonormal system. If we multiply $P u_i = \lambda_i u_i$ by $P$ from left on both sides, we have
$$P^2 u_i = \lambda_i P u_i = \lambda_i^2 u_i.$$
If $P$ is idempotent, the left side is $P u_i = \lambda_i u_i$. Hence, $\lambda_i = \lambda_i^2$, i.e., $\lambda_i = 0, 1$. Let $\lambda_1 = \cdots = \lambda_r = 1$, $\lambda_{r+1} = \cdots = \lambda_n = 0$. Then,
$$P u_i=u_i, i = 1, \dots, r,$$
$$P u_i = 0, i = r+1, \dots, n.$$
We see that $P$ is the orthogonal projection matrix onto the subspace spanned by $u_1, \dots, u_r$.
I cannot understand this statement:
As is well known, an $n \times n$ symmetric matrix $P$ has $n$ real eigenvalues $\lambda_1, \dots, \lambda_n$, the corresponding eigenvectors $u_1, \dots, u_n$ being an orthonormal system.
- I know that an $n \times n$ symmetric matrix $P$ has $n$ real eigenvalues $\lambda_1, \dots, \lambda_n$.
- And I know if $\lambda_i \ne \lambda_j$, then $u_i$ is orthogonal to $u_j$.
- And I know that every $n \times n$ symmetric matrix $P$ doesn't have distinct $n$ real eigenvalues.
Let $P$ be an $n \times n$ symmetric matrix $P$.
Are there eigenvecotrs of $P$ that are mutually orthogonal?
Best Answer
Yes, for any symmetric matrix $P$ of size $n$, it is possible to find a set of eigenvectors $(u_i)_{i=1}^n$ such that if $i\neq j$, then $u_i . u_j = 0$ (scalar product), and $||u_i||^2 = 1$ for all $i$.
Basically, a symmetric matrix always admits a set of eigenvectors which is an orthonormal basis of the vector space.
In matrix notations, it is possible to find an orthogonal matrix $O$ and a diagonal matrix $D$ (whose diagonal are the eigenvalues of $P$) such that $P = O D O^T = O D O^{-1}$.