Matrix-by-vector derivative Kronecker product

derivativeskronecker productlinear algebramatrix-calculus

We use $\otimes$ for the Kronecker. For $n \in \mathbb{N}$, let $F,G: \mathbb{R}^n \to \mathbb{R}^{n}$ be smooth functions. In this case, we have
$F(x)\otimes G(x) \in \mathbb{R}^{n^2}$

If $D$ stands for the total derivative operator then
$D(F(x)\otimes G(x))$ is an $n \times n^2$ matrix. Is it true that
$$D(F(x)\otimes G(x)) = DF(x) \otimes G(x) + F(x) \otimes DG(x)?$$

If so, are there any good reference for this particular fact?

Best Answer

Yes, I think this is true. It of course depends on how you order your basis of $\Bbb{R}^n \otimes \Bbb{R}^n$ to identify it with $\Bbb{R}^{n^2}$, and likewise, how you choose a basis of $\Bbb{R}^{n^2} \otimes \Bbb{R}^n$ to identify it with $n^2$-by-$n$ matrices.

I will assume that if $e_1,\dots,e_n$ is a basis of $\Bbb{R}^n$, then you are ordering your basis of $\Bbb{R}^n \otimes \Bbb{R}^n$ as $e_1 \otimes e_1$, $e_1 \otimes e_2$, $\dots$, $e_1 \otimes e_n$, $e_2 \otimes e_1, \dots, e_n \otimes e_1, \dots, e_n \otimes e_n$. So when you write $D(f \otimes g)$ as a matrix, I will assume the rows are ordered according to this basis.

So the entry of $D(f \otimes g)$ in the row indexed by $e_i \otimes e_j$ and column indexed by $e_k$ will be $$ \frac{\partial f_i}{\partial x_k} g_j + f_i \frac{\partial g_j}{\partial x_k} $$ Every entry of the matrix looks like this, so $D(f \otimes g)$ is the sum of two matrices. Look for now just at the left term $\frac{\partial f_i}{\partial x_k} g_j$. You can see these are the entries of $D(f) \otimes g$. Similarly, the right term are the entries of $f \otimes D(g)$.

As an example when $n=2$, let $f(x,y) = (f_1(x,y), f_2(x,y))$, and similarly for $g$. Then

$$ D(f) \otimes g = \left( \begin{array}{cc} \frac{\partial f_1}{\partial x} g_1 & \frac{\partial f_1}{\partial y} g_1 \\ \frac{\partial f_1}{\partial x} g_2 & \frac{\partial f_1}{\partial y} g_2 \\ \frac{\partial f_2}{\partial x} g_1 & \frac{\partial f_2}{\partial y} g_1 \\ \frac{\partial f_2}{\partial x} g_2 & \frac{\partial f_2}{\partial y} g_2 \end{array} \right) $$

And similarly, $$ f \otimes D(g) = \left( \begin{array}{cc} f_1 \frac{\partial g_1}{\partial x} & f_1 \frac{\partial g_1}{\partial y} \\ f_1 \frac{\partial g_2}{\partial x} & f_1 \frac{\partial g_2}{\partial y} \\ f_2 \frac{\partial g_1}{\partial x} & f_2 \frac{\partial g_1}{\partial y} \\ f_2 \frac{\partial g_2}{\partial x} & f_2 \frac{\partial g_2}{\partial y} \end{array} \right) $$

If you add them together, you get the matrix for $D(f \otimes g)$ if you order the coordinates so that $f \otimes g = (f_1g_1, f_1g_2, f_2g_1, f_2g_2)$.