I quote paragraph 2.5 of The Matrix Cookbook document: Assume $F(X)$ to be a differentiable function of each of the elements of $X$… $f(\cdot)$ is the scalar derivative of $F(\cdot)$. $X$ is here a matrix.
What is the scalar derivative? It is not defined in this document and I have issues to find a definition using Mister Google.
But the way, I'm puzzled by formula (100) of that document:
$$\frac{\partial}{\partial X} \mathsf{Tr}(XA) = A^T$$
$X \mapsto {Tr}(XA)$ is a linear form defined on the matrices vector space and therefore it's derivative is itself everywhere
$$\frac{\partial}{\partial X} \mathsf{Tr}(XA).H = \mathsf{Tr}(HA)$$
What is the link with $A^T$?
Best Answer
The simplest explanation is that the word $scalar$ is a typo.
The formula itself seem correct. For instance, let $$\eqalign{ F(x) &= \sin(x) \cr f(x) &= \frac{dF}{dx} = \cos(x) \cr }$$ Then, for a matrix argument $A$, one has the result $$\eqalign{ \frac{\partial\,{\rm Tr}(\sin(A))}{\partial A} &= \cos(A)^T \cr }$$ ...or $\cos(A)$ depending on which layout convention you prefer.