If $A$ and $B$ are orthogonal projection matrices, how can I show that trace$(AB) \le $ rank$(AB)$

inequalitylinear algebramatricesmatrix-ranktrace

If $A$ and $B$ are orthogonal projection matrices, how can I show that trace$(AB) \le $ rank$(AB)$?

I was using C-S inequality to get tr$(AB) \le \sqrt{tr(A^2)tr(B^2)}$ and I know that $tr(A^2)=$rank$(A)$. But I can't get the rank of $AB$.

Best Answer

If either $A$ or $B$ is zero, this holds trivially.

Suppose that both $A$ and $B$ are non-zero. It suffices to show that all eigenvalues of $AB$ have magnitude at most equal to $1$. To that end, note that if $\|\cdot\|$ denotes the spectral norm, then we have $\|A\| = \|B\| = 1,$ so that $\|AB\| \leq \|A\| \cdot \|B\| = 1$. It follows that all eigenvalues $\lambda$ of $AB$ satisfy $|\lambda| \leq \|AB\| \leq 1$. Thus, if $AB$ has rank $r$ and $\lambda_1,\dots,\lambda_k$ (with $k \leq r$) are the non-zero eigenvalues of $AB$, then we have $$ \operatorname{tr}(AB) \leq |\operatorname{tr}(AB)| = \left|\sum_{i=1}^k\lambda_i\right| \leq \sum_{i=1}^k|\lambda_i| \leq \sum_{i=1}^k 1 = k \leq r, $$ which is what we wanted.