I am struggling with the following derivative. Let $\pmb{x} \in \mathbb{R}^{n}$ be a vector, $\pmb{y} \in \mathbb{R}^{m}$ another vector that is a function of $\pmb{x}$, and $\pmb{g}$ and $\pmb{h}$ two functions returning vectors. I aim to obtain the following derivative:
$$
\frac{\partial}{\partial \pmb{x}}
\left( \pmb{g}(\pmb{y})^{\top} \otimes \pmb{I} \right)\pmb{h}(\pmb{x}).
$$
My thoughts were that this would involve a combination of the sum and chain rule. E.g., first take the sum-rule with the left and right-hand part. But then $\left( \pmb{g}(\pmb{y})^{\top} \otimes \pmb{I} \right)$ takes the form of a matrix and I am quite lost. I expect that taking a differential form is needed, but I don't really understand how to do these properly yet.
Best Answer
Define some new matrices $$\eqalign{ H &= {\rm Mat}(h) \implies h = {\rm vec}(H) \cr J &= \frac{\partial h}{\partial x},\quad K = \frac{\partial g}{\partial y},\quad L = \frac{\partial y}{\partial x} \cr }$$ Write the vector function and find its differential and gradient in terms of these new variables. $$\eqalign{ w &= (g^T\otimes I)h = Hg \cr dw &= (g^T\otimes I)\,dh + H\,dg \cr &= \Big((g^T\otimes I)J + HKL\Big)\,dx \cr \frac{\partial w}{\partial x} &= (g^T\otimes I)J + HKL \cr }$$ Knowing nothing about the nature of the vector functions $(g,h,y)$ this is as far as we can go.