I have an $n \times r$ matrix $A$ and an $r \times m$ matrix $B$.
What will the derivative of $ABB^T$ with respect to $A$ be? Is it simply $BB^{T}$?
Matrix differentiation with respect to matrix
derivativesmatricesmatrix-calculus
derivativesmatricesmatrix-calculus
I have an $n \times r$ matrix $A$ and an $r \times m$ matrix $B$.
What will the derivative of $ABB^T$ with respect to $A$ be? Is it simply $BB^{T}$?
Best Answer
Consider the matrix-valued function
$$\mathrm F (\mathrm X) := \mathrm X \mathrm C$$
The $(i,j)$-th entry of the output is
$$f_{ij} (\mathrm X) := \mathrm e_i^\top \mathrm F (\mathrm X) \,\mathrm e_j = \mathrm e_i^\top \mathrm X \mathrm C \,\mathrm e_j = \mathrm e_i^\top \mathrm X \,\mathrm c_j = \mbox{tr} \left( \mathrm c_j\mathrm e_i^\top \mathrm X \right) = \langle \mathrm e_i\mathrm c_j^\top, \mathrm X \rangle$$
where $\mathrm c_j$ is the $j$-th column of $\mathrm C$ and $\langle \cdot, \cdot \rangle$ denotes the Frobenius inner product. Hence, the gradient of $f_{ij}$ with respect to $\mathrm X$ is
$$\nabla f_{ij} (\mathrm X) = \mathrm e_i\mathrm c_j^\top$$