Learning Linear Algebra on my own time. Came upon a problem, which asked to find a projection matrix $P$ onto a column space of $A = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0\end{bmatrix}$ and project vector $b = \begin{bmatrix} 1 \\ 2 \\ 3 \\ 4\end{bmatrix}$ onto it.
The solution if fairly straight forward and the answer is $P = \begin{bmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0\end{bmatrix}$ and $Pb = \begin{bmatrix} 1 \\ 2 \\ 3 \\ 0\end{bmatrix}$.
There is another question to this problem.
For the matricies $A$ and $P$, if $P$ projects onto the columsn space of $A$ then what does $I-P$ projects onto?
Intuition told me that it is projecting onto left nullspace of $A$, i.e. onto the nullspace of $A^T$. And indeed it is not hard to demonstrate. Simply do the $I-P$ calculation and verify that it projects onto a left nullspace of $A$.
Here is my question. Say I have a generic matrix $A$ with its projection matrix $P$ that projects onto a column space of $A$. Is it always the case that $I-P$ will project onto a left nullspace of $A$? If it is, then how do I prove it? If it isn't, are there certain conditions that this will make it true? How do I find those conditions?
Best Answer
Yes, that is true in general.
First, note that by definition the left nullspace of $A$ is the orthogonal complement of its column space (which, by the way, is unique, and so we say "the column space of $A$" rather than "a column space"), because $A^T x = 0$ if and only if $x$ is orthogonal to every column of $A$.
Therefore, if $P$ is an orthogonal projector onto its column space, then $I - P$ is a projector onto its orthogonal complement, i.e., the nullspace of $A^T$. To see this, first note that, by definition, $Px = x$ for all $x$ is in the column space of $A$. Thus,
$(I - P)x = x - P x = x - x = 0$.
On the other hand, if $y$ is in the left nullspace of $A$, then $P y = 0$, and so
$(I - P)y = y - Py = y - 0 = y$.
Edit: also, if $P$ is an orthogonal projector, it is self-adjoint, and so is $I-P$, because the sum of two self-adjoint linear operators is also self-adjoint. Hence, in that case, $I-P$ is also an orthogonal projector.