How can I find the gradient of the term $a^TXb$ where $X$ is a $n \times m$ matrix, and $a$ and $b$ are column vectors. Since the gradient is with respect to a matrix, it should be a matrix. But I do not have a clue on how to derive this gradient.
Any help ?
Best Answer
Write the function in terms of the inner/Frobenius product (which I'll denote by a colon). Then finding the differential and gradient is straightforward $$\eqalign{ f &= ab^T:X \cr\cr df &= ab^T:dX \cr\cr \frac{\partial f}{\partial X} &= ab^T \cr\cr }$$ Note that the inner product is really just an infix notation for the trace $$A:B = {\rm tr}(A^TB)$$