[Math] Show that $P$, an $n \times n$ matrix, is a projection given only that $P^2 = P$, and $P^T = P$

Question

Let $P$ be a $n \times n$ matrix such that $P^2 = P$ and $P^T = P$. Show that $P$ is the projection onto a subspace of $\mathbb{R}^n$. Hint: If P is indeed a projection, the subspace P is projecting onto must be ??. Show that $\overrightarrow{b} – P\overrightarrow{b}$ is orthogonal to ??.

Question: According to the professor, showing that all vectors in the column space of $P$ being orthogonal to $\overrightarrow{b} – P\overrightarrow{b}$ is enough to show that P is a projection. Why is this true? And how do I show it?

Steps I've Taken: I've tried constructing a matrix $A$ such that it's column space forms a basis for a subspace onto which $P$ supposedly projects, but I can't find how to leap from that fact to proving orthogonality. Also, I don't believe this is the right approach, since it isn't using the column space of $P$.

Answer 1

Your question is not well clear, but I'm attempting an answer.

First note that the usual definition is that

a linear transformation $P$ is a projector if $P^2=P$

Your additional condition $P^T=P$ caracterize the orthogonal projectors (in a real vector space $V$), where:

$P$ is orthogonal if the kernel of $P$ is orthogonal to its range.

Now note that for any vector $y \in V$ we have that $(y-Py)$ is an element of $\ker(P)$ because $P(y-Py)=Py -P^2y=0$.

So, $P$ is an orthogonal projector if $\forall x,y \in V$ we have : $$ \langle Px,(y-Py) \rangle=0 $$ But this is done because: $$ \langle Px,(y-Py) \rangle=\langle P^2x,(y-Py) \rangle=\langle Px,P^T(y-Py) \rangle=\langle Px,(Py-P^2y) \rangle=\langle Px,0 \rangle=0 $$