Calculus – Derivative of a Matrix with Respect to a Vector

calculusderivativeslinear algebramatrices

I know that for two k-vectors, say $A$ and $B$, $\partial A/\partial B$ would be a square $k \times k$ matrix whose $(i,j)$-th element would be $\partial A_i/\partial B_j$.

But could someone please explain how the partial derivative look like if we were differentiating $k \times k$ matrix instead? That is, $M$ is a $k \times k$ matrix, $x$ is a $k$-vector, how can we write $\partial M/\partial x$?

I tried to use the first principles, but no luck so far..

Thanks

Best Answer

A natural definition is $$\frac{\partial {\textbf A}}{\partial {\textbf x}}={\textbf C}$$ where C is a 3D matrix (or tensor) with $$C_{n,m,\ell}=\frac{\partial A_{n,m}}{\partial x_\ell}$$

The order of the subindexes is open to different conventions, of course.