Assume $A$ is nonsingular and $X$ is symmetric, I want to find this derivative
\begin{align}
&\frac{\partial \operatorname{trace}(AX^{-1}A^TX)}{\partial X}.
\end{align}
In matrix cookbook I found this:
For symmetric $C$ we have
\begin{align}
&\frac{\partial \operatorname{trace}((X^TCX)^{-1}A)}{\partial X}=-(CX(X^TCX^{-1}))(A+A^T)(X^TCX)^{-1}.
\end{align}
Assuming that $X,C$ are nonsingular, and $C=A^{-T}$, $\operatorname{trace}((X^TCX)^{-1}A)=\operatorname{trace}(AX^{-1}A^TX^{-T})$, unfortunately a little bit different then what I want.
Best Answer
Rewrite the objective function as $$\eqalign{ \phi &= AX^{-1}:XA \\ }$$ where the colon denotes the trace/Frobenius product, i.e. $$A:B = {\rm Tr}(A^TB) = {\rm Tr}(AB^T)$$ Then calculate the gradient $$\eqalign{ d\phi &= AX^{-1}:dX\,A + XA:A\,dX^{-1} \\ &= AX^{-1}A^T:dX + A^TXA:dX^{-1} \\ &= AX^{-1}A^T:dX - A^TXA:\big(X^{-1}\,dX\,X^{-1}\big) \\ &= \big(AX^{-1}A^T - X^{-1}A^TXAX^{-1}\big):dX \\ \frac{\partial \phi}{\partial X} &= AX^{-1}A^T - X^{-1}A^TXAX^{-1} \\ }$$ The cyclic property of the trace allow the terms in a Frobenius product to be rearranged in a number of equivalent ways, i.e. $$\eqalign{ &A:B = B:A = B^T:A^T \\ &A:BC = AC^T:B = C^T:A^TB \\ }$$