Given a real rectangular matrix $X$, and two scalar-valued matrix functions, $f(X)$ and $g(X)$, does the product rule for differentiation of a product of scalar valued functions, hold when differentiating the product, $f(X)g(X)$ w.r.t $X$ ? If not, what would be the corresponding product rule? Let's assume that the product $f(X)g(X)$ gives a real valued scalar, and is well-defined in terms of the dimensions.
Note: $f(.)$ and $g(.)$ can be the matrix trace function for example.
Best Answer
The product rule holds in very great generality. Let $X,Y,Z,W$ be Banach spaces with open subset $U \subset X$, and suppose $f: U \rightarrow Y$ and $g: U \rightarrow Z$ are Frechet differentiable. If $B(\cdot, \cdot): Y \times Z \rightarrow W$ is a continuous bilinear map, then for any $\xi \in X$,
$$ \frac{d}{dx}[ B(f(x), g(x))](\xi) = B(f'(x)\xi, g(x)) + B(f(x), g'(x)\xi)$$
where all the derivatives in question are Frechet derivatives. To apply to your case, we take $U = X = \mathbb{R}^{n \times n}$, $Y =Z = W = \mathbb{R}$, and $B(y,z) = yz$.