[Math] Proving Positive Definiteness of Symmetric Block Matrix

block matricesinner-productslinear algebramatricespositive definite

I'd like to show that the block matrix $X = \pmatrix{L_0 + L_1 & h L_1 \\ h L_1 & h L_1}$ is positive definite (or has non-zero determinant, or no eigenvalues equal to zero), where $L_i = J_i^\mathsf{T} J_i$ and the scalar $h<1$ (if that's important). I'm assuming that $J_i$ has full column rank (more rows than columns). Here are three approaches I've examined. Please comment on a good way to prove that $X$ is positive definite.

  1. Block column operations to get a triangular matrix. Column 1 is replaced by (Column 1 – Column 2). $X = \pmatrix{L_0 + L_1 – h L_1 & h L_1 \\ 0 & h L_1}$. The determinant of this is det$\left[L_0 + (1-h) L_1\right]\cdot$det$(hL_1)$, which is greater than zero.
  2. From Wikipedia on "Schur complement condition for positive definiteness": $X$ is positive definite if and only if $h L_1$ and $L_0 + L_1 – h L_1 (h L_1)^{-1} (h L_1)$ are both positive definite. The problem is that I do not know this to be so other than that Wikipedia says it is. That is, I think it's easy to show that both are positive definite, but I do not know how this assures that $X$ is positive definite.
  3. (Again from Wikipedia:) The Schur complement of $A = L_0 + L_1$ in $X$ is $S = h L_1 – h L_1 (L_0 + L_1)^{-1} (h L_1)$. "$X$ is positive definite if and only if $A$ and $S$ are both positive definite." I have been unable to prove that $S$ is positive definite. Here's my attempt. The term $(L_0 + L_1)^{-1}$ in $S$ becomes $L_1^{-1} – L_1^{-1} \left( L_0^{-1} + L_1^{-1} \right)^{-1} L_1^{-1}$ by the Woodbury matrix identy. Substituting this into $S$ yields $S = h\left\{L_1 – h L_1 \left[ L_1^{-1} – L_1^{-1} \left( L_0^{-1} + L_1^{-1} \right)^{-1} L_1^{-1} \right] L_1 \right\}$, which reduces to $S = h\left\{ L_1 + h \left[ \left(L_0^{-1} + L_1^{-1}\right)^{-1} – L_1\right]\right\}$. That does not necessarily look positive definite to me, what with the difference in there.

Best Answer

Presumably all matrices here are real and $h>0$. As each $L_i=J_i^\top J_i$ for a full-rank tall matrix $J_i$, it is positive definite. Now consider a real vector $x^T=(u^T,v^T)$. Then \begin{align*} x^TXx =&u^\top (L_0+L_1)u + hu^\top L_1v + hv^\top L_1u + hv^\top L_1v\\ =&u^\top L_0u + (1-h)u^\top L_1u + h(u+v)^\top L_1(u+v).\tag{2} \end{align*} As $L_1$ and $L_2$ are positive definite and $h>0$, the quantity in $(2)$ is nonnegative. Also, it is zero if and only if each summand is zero, i.e. iff $u=u+v=0$, or $x=0$. Therefore $X$ is positive definite.

The above proof shows that one can relax the condition on $h$ so that it can be equal to $1$. However, when $h\le 0$ or $h>1$, $X$ is not necessarily positive definite. Here are some counterexamples:

  • $h=0,\,L_0=1,\,J_1=\pmatrix{1\\ 0}$ and $X=\pmatrix{1&0\\ 0&0}$.
  • $h=-1,\,L_0=1,\,J_1=\pmatrix{1\\ 0}$ and $X=\pmatrix{2&-1\\ -1&-1}$.
  • $h=2,\,L_0=1,\,J_1=\pmatrix{1\\ 0}$ and $X=\pmatrix{2&2\\ 2&2}$.
Related Question