I am trying to compute the gradient with respect to a vector $\mathrm x \in \mathbb{R}^d$ of a complicated expression involving the trace of matrix product.
The expression is the following:
$$ F(\mathrm x) = \text{Tr}\left[ C \nabla (C^{-1} \mathrm f(\mathrm x)) \right],$$
where $C$ is a $d \times d$ matrix, and $\mathrm f(\mathrm x): \mathbb{R}^d \rightarrow \mathbb{R}^d$ a vector valued function.
What I have until now is
$$ \nabla F(\mathrm x) = \nabla \text{Tr}\left[ C (C^{-1} J_f(\mathrm x)) \right],$$
where with $J_f(\mathrm x)$ I denote the Jacobian of $\mathrm f$.
However I do not know how to proceed and take the gradient of the trace.
Would it be like this? $$\nabla \text{Tr}\left[ C (C^{-1} J_f(\mathrm x)) \right] = \text{Tr}\left[\nabla\left( C (C^{-1} J_f(\mathrm x)) \right)\right]$$
Best Answer
Unfortunately, your second equation is incorrect.
In index notation, the function $F$ is $$\eqalign{ \def\n{\nabla} \def\p{\partial} \def\T{\operatorname{Tr}} F &= C_{ij}\;\p_j(C_{ik}^{-1} f_k) \\ &= C_{ij} C_{ik}^{-1}\;(\p_j f_k) \\ &= C_{ij} C_{ik}^{-1}\:J_{jk} \\ &= C_{ki}^{-T}C_{ij}\:J_{jk} \\ }$$ which can be expressed in terms of the trace in various ways $$\eqalign{ F &= \T(C^{-T}C\:J) \\ &= \T(J^TC^TC^{-1}) \\ }$$ But the gradient $\n F$ is $$\eqalign{ \p_\ell F &= C_{ij} C_{ik}^{-1}\;(\partial_\ell J_{jk}) }$$ which doesn't have a simple trace expression.
The biggest obstacle is that the gradient is a vector while the trace is a scalar.
$\sf NB\!:\:$ If $\,C=C^T$ then you can proceed as indicated in gunes answer