$\newcommand{\R}{\operatorname{Ran}} \newcommand{\K}{\operatorname{Ker}}\newcommand{\b}{\mathbf}$
Prove that a self adjoint and idempotent matrix $P$ is an orthogonal projection matrix.
I was given the following hint:
Consider the decomposition $\b x = \b x_1 + \b x_2$, where $\b x_1 \in \R P$, and $\b x_2 ⊥ \R P$. Now show that $P \b x_1 =\b x_1$, and $P\b x_2 = \b 0$.
I proved the $2$nd condition of the hint as follows.
Since $\b x_2 \in (\R P)^\perp = \K P^* = \K P$, $P \b x_2 = 0$.
I think the first hint is not correct. I think it should be $\b x_1 – P \b x_1 \in \K P = (\R P)^\perp$, which is easy enough to prove. Am I right ?
Best Answer
If $P$ is an orthogonal projection matrix, it is with respect to its column space (range). Let $U$ be the column space of $P$: $$ U=\{Px:x\in\mathbb{R}^n\} $$ Take any $u\in U$; then $u=Pv$ for some $v$ and therefore $u=Pv=P^2v=P(Pv)=Pu$.
If $w\in U^\perp$, then $\langle w,u\rangle=0$, for every $u\in U$; it follows that, for every $u\in U$, $$ \langle Pw,u\rangle=w^TP^Tu=w^TPu=w^Tu=0 $$ so that $Pw\in U^\perp$.
Now write $x\in\mathbb{R}^n$ as $x=y+z$, with $y\in U$ and $z\in U^\perp$; then, by definition, $y$ is the orthogonal projection of $x$ onto $U$ and $$ Px=Py+Pz=y+Pz $$ with $Pz\in U^\perp$, as showed before. Since the decomposition of a vector as sum of a vector in $U$ and a vector in $U^\perp$ is unique, from the fact that $Px\in U$ it follows that $y=Px$ and $Pz=0$.
Therefore the orthogonal projection of $x$ onto $U$ is $y=Px$.