The Matrix Cookbook says that:
$\frac{\partial}{\partial\mathbf{X}} Tr\{\mathbf{A}\mathbf{X}\mathbf{B}\} = \mathbf{A^T}\mathbf{B^T}$
I can't seem to get this. I know that:
$\frac{\partial}{\partial\mathbf{X}} Tr\{F(\mathbf{X})\} = f(\mathbf{A}\mathbf{X}\mathbf{B})^T$
Where $f$ is the scalar derivative of $F$.
So when I apply the rule: $\partial \mathbf{XY} = \partial \mathbf{X} \mathbf{Y} + \mathbf{X} \partial \mathbf{Y}$
I do:
Let: $\mathbf{C} = \mathbf{X}\mathbf{B}$
Then: $\mathbf{A}\mathbf{X}\mathbf{B} = \mathbf{A}\mathbf{C}$.
$\frac{\partial}{\partial\mathbf{X}} \mathbf{A}\mathbf{C} = \frac{\partial}{\partial\mathbf{X}} \mathbf{A} \mathbf{XB} + \mathbf{A} \frac{\partial}{\partial\mathbf{X}}\mathbf{X}\mathbf{B} = \mathbf{AB}$
But then:
$(\mathbf{AB})^{T} = \mathbf{B^TA^T} \neq \mathbf{A^{T}B^{T}}$
Am I confusing the notion of scalar derivative and matrix derivative? How can I verify the Cookbook's claim?
Best Answer
The matrix inner product (denoted by a colon) is equivalent to the trace $$A^T:B = {\rm tr}(AB)$$
Therefore $$\eqalign{ f &= {\rm tr}(AXB) \cr &= {\rm tr}(BAX) \cr &= (BA)^T:X \cr &= A^TB^T:X \cr \cr df &= A^TB^T:dX \cr \cr \frac{\partial f}{\partial X} &= A^TB^T \cr }$$