I need to compute the derivative of a matrix by a vector and I'm using matrix vectorization to do so. Define this derivative as $$ \frac{\partial vec[G(x)]}{\partial vec[(x^T)]} $$ where $x \in \mathbb{R^n}$ and $G(x) \in \mathbb{R}^{m,p}$. My matrix is partitioned as follows
$$G(x)=\begin{bmatrix}
0 \\
A(x) \\
0
\end{bmatrix}$$
where each partition is $ \mathbb{R}^{n,p}$ and thus $m = 3n$. Now, it is obvious that
$$ vec[G(x)] \neq \begin{bmatrix}
vec[0] \\
vec[A(x)] \\
vec[0]
\end{bmatrix} $$
and thus I cannot compute the derivative by taking the derivative of $vec[A]$ and pad with zeros. Is there any identity or property which I can use to simplify my calculation?
Thanks
Best Answer
$ \def\bbR#1{{\mathbb R}^{#1}} \def\e{\varepsilon}\def\o{{\tt1}}\def\p{\partial} \def\L{\left}\def\R{\right} \def\LR#1{\L(#1\R)} \def\vecc#1{\operatorname{vec}\LR{#1}} \def\trace#1{\operatorname{Tr}\LR{#1}} \def\qiq{\quad\implies\quad} \def\grad#1#2{\frac{\p #1}{\p #2}} \def\c#1{\color{red}{#1}} \def\m#1{\left[\begin{array}{r}#1\end{array}\right]} $The rule for vectorizing Kronecker products is $$\eqalign{ m,n &= {\rm size}(Y) \qquad\quad p,q = {\rm size}(X) \\ {\rm vec}(Y\otimes X) &= (I_n\otimes K_{q,m}\otimes I_p)\,\Big({\rm vec}(Y)\otimes I_q\otimes I_p\Big)\,{\rm vec}(X) \\ {\rm vec}(X\otimes Y) &= (I_q\otimes K_{n,p}\otimes I_m)\,\Big(I_q\otimes I_p\otimes{\rm vec}(Y)\Big)\,{\rm vec}(X) \\ }$$ where $I_n\in\bbR{n\times n}$ is the identity matrix and $K_{q,m}\in\bbR{qm\times qm}\,$ is the Commutation Matrix.
Given the standard basis vectors $\{\e_k\in\bbR{3}\}$ you can write the structured matrix as a Kronecker product $$\eqalign{ G &= \LR{\e_2\otimes A} \\ }$$ and then apply the vectorization rule to obtain $\;{\rm vec}(G)=M\cdot{\rm vec}(A)$