Linear Algebra – Orthogonal Projection Matrix

linear algebramatrices

Let $A\in M_{m\times n}(\mathbb{R})$. Denoting by $R(A)$ the column space of $A$ and $N(A)$ the null space of $A$.

I know that

$z^*=Ax^*$ is a projection of $b\in R^m$ on $R(A)=N(A^T)$ where $x^*=Ab$

$r^*=b-z^*$ is a projection of $b\in R^m$ on $N(A^T)=R(A)^\perp$

Similarly

$z^*=A^Tx^*$ is a projection of $b\in R^n$ on $R(A^T)=N(A)^\perp$ where $x^*=A^TAx=Ab$

$r^*=b-z^*$ is a projection of $b\in R^n$ on $N(A)=R(A^T)^\perp$

Finally I know the orthogonal projection matrix on the subspace $R(A)$ is defined by $P=A(A^TA)^{-1}A^T$.

my question is how do I determine for example the projection matrix on $R(A^T),N(A^T),N(A)$?

EDIT:* means nothing is just notation that I used

Best Answer

If $A^+$ is the Moore-Penrose pseudo inverse of $A$, that is, the matrix such that $$\tag{1} a)\;AA^+A=A, \quad b)\;A^+AA^+=A^+, \quad c)\;(AA^+)^*=AA^+, \quad d)\;(A^+A)^*=A^+A,$$ then

  • (i) $AA^+$ is the orthogonal projector (OP) onto the range of $A$,
  • (ii) $A^+A$ is OP onto the range of $A^*$.

Note that if $Q$ is OP onto a subspace $S$, that is, $Q=Q^*$ (here, $^*$ denotes conjugate transpose), $Q^2=Q$, and $R(Q)=S$ (note that these three conditions imply that $Q$ is unique), then $I-Q$ is OP onto $S^\perp$. So, if (i) is true, then $I-AA^+$ is OP onto $R(A)^\perp=N(A^*)$ and, if (ii) is true, then $I-A^+A$ is OP onto $R(A^*)^\perp=N(A)$.

For the proof of (i) and (ii), we use only (1) and the fact that if $X=YZ$, then $R(X)\subseteq R(Y)$. We need only to show that the matrices $AA^+$ and $A^+A$ are Hermitian, idempotent, and their ranges are equal to the subspaces on which they are supposed to project.

Both $AA^+$ and $A^+A$ are obviously Hermitian; see (1c) and (1d). In addition, (1a) and/or (1b) imply that they are idempotent. It remains to show that $R(AA^+)=R(A)$ and $R(A^+A)=R(A^*)$. Clearly, $R(AA^+)\subseteq R(A)$; $R(A)\subseteq R(AA^+)$ follows from (1a). From (1d), we have $A^+A=A^*(A^+)^*$, so $R(A^+A)\subseteq R(A^*)$. From (1a) and (1d), $A^*=A^+AA^*$, so $R(A^*)\subseteq R(A^+A)$.

Summarizing, the following are OPs:

  • $AA^+$ on $R(A)$,
  • $I-AA^+$ on $N(A^*)$,
  • $A^+A$ on $R(A^*)$,
  • $I-A^+A$ on $N(A)$.

Note that this is true no matter what is the rank of $A$. While for some $A$, $A^*A$ or $AA^*$ (or both) might not be invertible depending on the rank of $A$, any $A$ has always a unique Moore-Penrose pseudoinverse and a set of unique OPs given above associated with the four subspaces of $A$.

Related Question