[Math] Trace of a Matrix Product.

analysislinear algebramatricestracevectors

Let $A \in \mathbb{R}^{n\times m}$, and $B\in \mathbb{R}^{m\times m}$. Let $A'$ denote the transpose of $A$.

From this we know that : $A'\in \mathbb{R}^{m\times n}$ and $AB\in \mathbb{R}^{n\times m}$ and hence $ABA' \in \mathbb{R}^{n\times n}$.

The Trace of a matrix is defined as the sum of its diagonal entries. $\textbf{My Question :}$

Does anyone know bounds (using any usual matrix norm) for

  • $\text{Trace}(ABA')$

  • Or more generally $\text{Trace}(AB)$

Best Answer

Whenever you see a matrix trace, you should think inner product, because

$$ \operatorname{Tr}(A^T B) = \langle A, B\rangle_F = \langle A,B\rangle_{\mathbb R^m \otimes \mathbb R^n}$$

that is, the trace of the product of two matrices is equal to their frobenius inner product, which in turn is the induced inner product on the tensor product of Hilbert spaces.

Since it is an inner product, the Cauchy-Schwartz inequality applies:

$$ |\langle A, B \rangle_F |^2 \le \|A\|_F^2\|B\|_F^2$$

with equality if and only if $A$ and $B$ are linearly dependent matrices, i.e. scaler multiples of each other. In your case, we have

$$ |\operatorname{Tr}(ABA^T)| = |\operatorname{Tr}(A^TA B)| = |\langle A^TA , B\rangle_F| \le \|A^TA\|_F\|B\|_F$$

The last term can be further bounded by

$$\begin{aligned} \|A^TA\|_F\|B\|_F &\le \|A\|_F^2\|B\|_F = \Big(\sum\nolimits_i \sigma_i^2(A)\Big)\cdot\sqrt{\sum\nolimits_j \sigma_j^2(B)} \\ &\le rank(A)\cdot \sigma^2_{\max}(A)\cdot rank(B)\cdot\sigma_{\max}(B)\\ &\le m\cdot \min(m,n)\cdot \sigma^2_{\max}(A)\cdot \sigma_{\max}(B) \end{aligned}$$