Suppose I have a block-partitioned matrix
$$\begin{bmatrix} \mathbf{X}_1^{\top}\mathbf{X}_1 & \mathbf{X}_1^{\top}\mathbf{X}_2 \\ \mathbf{X}_2^{\top}\mathbf{X}_1 & \mathbf{X}_2^{\top}\mathbf{X}_2 \end{bmatrix}$$
where $\mathbf{X}_1$ is $G \times K_1$ and $\mathbf{X}_2$ is $G \times K_2$. Thus, our block-partitioned matrix is equal to the outer product
$$\begin{bmatrix} \mathbf{X}_1^{\top} \\ \mathbf{X}_2^{\top} \end{bmatrix}\begin{bmatrix} \mathbf{X}_1 & \mathbf{X}_2 \end{bmatrix}$$
and its dimension is $K \times K$ where $K=K_1+K_2$. In general, when is this block-partitioned matrix invertible? Is there a necessary and sufficient condition?
Edit. My first thought is: for the block-partitioned matrix to be invertible, it is equivalent that each of the four blocks are invertible. However, I no longer think this. For example, if one of the off diagonals, say the lower left, $\mathbf{X}_2^{\top}\mathbf{X}_1=\mathbf{0}$, the matrix could still be invertible if the blocks on the diagonal are. I can't articulate a proof of that though — could anyone please help?
Best Answer
The matrix $$ Y:=X^TX, \quad X:=[X_1,X_2], $$ (which is generally positive semidefinite) is invertible iff $[X_1,X_2]$ has full column rank.
So necessarily, $X_1$ must have full column rank. However, full rank of $X_2$ is not sufficient for the nonsingularity of $Y$. From the block inversion formulas it follows that
The matrix $P_1:=I-X_1(X_1^TX_1)^{-1}X_1^T$ is an orthogonal projector onto the complement of the range of $X_1$, that is, the nullspace of $X_1^T$. Since $X_2^T(I-X_1(X_1^TX_1)^{-1}X_1^T)X_2=(P_1X_1)^T(P_1X_1)$, we have that (1) is invertible iff $P_1X_1$ has full column rank, that is, the columns of $X_1$ do not become dependent when projected onto the nullspace of $X_1^T$.
Note that the same argument can be made with the indices $1$ and $2$ exchanged.