Derivative including kronecker product

chain rulekronecker productmatrix-calculus

Suppose that $\delta$ is a $p\times1$ vector and $M$ is a symmetric $Np\times Np$ matrix.

I'm trying to differentiate $\ln L=\ln|(I_{N}\otimes\delta')M(I_{N}\otimes\delta)|$ with respect to $\delta$ , where $|A|$ is the determinant of the matrix $A$.

I know that $\frac{\partial \ln L}{\partial(I_{N}\otimes\delta)}=2M(I_{N}\otimes\delta)[(I_{N}\otimes\delta')M(I_{N}\otimes\delta)]^{-1}$ given that $M$ is symmetric.

How should I apply the chain rule to get $\frac{\partial \ln L}{\partial \delta}$?

Best Answer

For typing convenience, define the following variables $$x=\delta,\qquad Y=I_N\otimes x,\qquad A = Y^TMY,\qquad\lambda=\ln(\det(A))$$ Assume you are given an analytic function of a scalar variable and its derivative $$f=f(s),\qquad p(s)=\frac{df}{ds}$$ These functions can be applied to a square matrix argument to yield the matrix values $$F=f(A),\qquad P=p(A)$$ The gradient (or equivalently the differential) of the trace of the matrix function can be written as $$\frac{\partial {\rm Tr}(F)}{\partial A} = P^T \quad\iff\quad d\,{\rm Tr}(F) = P^T:dA$$ In this particular case, the scalar functions that will be needed are $$f(s)=\ln(s),\qquad p(s)=s^{-1}$$ and the matrices are symmetric.

Therefore $$\eqalign{ d\lambda &= d\,\ln(\det(A)) \\ &= d\,{\rm Tr}(\ln(A)) &\quad\big({\rm Jacobi\,formula}\big) \\ &= A^{-1}:dA \\ &= A^{-1}:(Y^TM\,dY+dY^TMY) \\ &= 2A^{-1}:Y^TM\,dY \\ &= 2MYA^{-1}:dY \\ &= 2MYA^{-1}:(I_N\otimes dx) &\quad\big({\rm Your\,formula}\big) \\ &= 2\;{\rm vec}\big(MYA^{-1}\big):\big({\rm vec}(I_N)\otimes I_p\big)\,dx\\ &= 2\,\big({\rm vec}(I_N)\otimes I_p\big)^T{\rm vec}\big(MYA^{-1}\big):dx\\ \frac{\partial\lambda}{\partial x} &= 2\,\big({\rm vec}(I_N)\otimes I_p\big)^T{\rm vec}\big(MYA^{-1}\big) \\ }$$


In several of the steps above, a colon is used to denote the trace/Frobenius product, i.e. $$\eqalign{ A:B = {\rm Tr}(A^TB) = {\rm Tr}(B^TA) = B:A }$$