I want to invert the following matrix:
$$
I_m – B (B^T B)^{-1} B^T
$$
where $B$ is $m \times n$ with $m > n$ and $B$ has full column rank. The Woodbury matrix inverse identity states:
$$
(A + UCV)^{-1} = A^{-1} – A^{-1} U (C^{-1} + V A^{-1} U)^{-1} V A^{-1}
$$
Now if we identify:
\begin{align}
A &= I_m \\
U &= B \\
C &= -(B^T B)^{-1} \\
V &= B^T
\end{align}
then we have
\begin{align}
(I_m – B (B^T B)^{-1} B^T)^{-1} &= I_m – B (-B^T B + B^T B)^{-1} B^T \end{align}
but this leads to the inverse of a singular matrix. So this seems to indicate that the Woodbury formula won't work on my matrix.
Does anyone know of another way to invert my matrix?
Best Answer
Consider the example: let \begin{align} B = \begin{pmatrix} 1 & 0\\ 0 & 1\\ 0 & 0 \end{pmatrix} \end{align} then \begin{align} B^TB = I_2 \end{align} which means \begin{align} B(B^TB)^{-1}B^T = BB^T= \begin{pmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & 0 \end{pmatrix}. \end{align} Hence it follows \begin{align} I-B(B^TB)^{-1}B^T = \begin{pmatrix} 0 & 0 & 0\\ 0 & 0 & 0\\ 0 & 0 & 1 \end{pmatrix} \end{align} which is not invertible.
In fact, $P=B(B^TB)^{-1}B^T$ is a projection matrix onto the column space of $B$ and $I-P$ projects onto the orthogonal complement.