We have a projection matrix, $P = A(A^T A)^{-1} A$. The columns of $A$, we're given, form a basis for some subspace $W$. I need to prove that for any vector in the orthogonal complement of $W$, if we act on it with this projection matrix, we get that vector.
I'm unsure even on where to start. I know the definition of the orthogonal complement (though I can't quite see how to implement this, as the proof does not involve, as far as I can tell, inner products), and that we can write any vector in the subspace as a linear combination of $A$, given the fact that $A$ is a basis for the subspace. We can also take the original space, say, $V$ (such that $W \subset V$) and write any vector as a sum of a vector in $W$ and its subspace. This last fact is the only possible starting point I can think of.
I'd greatly appreciate any insights on this, particularly on how to start the proof, as I'd very much like to work through it and figure it out. Thanks in advance.
Best Answer
Note that since $P$ is the projection matrix onto W, for any vector in the orthogonal complement of $W$, if we act on it with this projection matrix, we get zero vector.
What you are claiming is true for the matrix $(I-P)$ that is the projection matrix for the orthogonal complement of $W$.
Refer to How to prove the complement $P^\perp$ of a projection matrix $P$ have relation $I-P=P^\perp$