I encounter a difficulty in one of my research dealing with derivative of a matrix. I will appreciate much for your expertise on this.
Let $\mathbf{x }=vec\left( X \right) $ where $X $ is a $T\times r
$ matrix of unknown parameters ($T$ and $r$ are constant, $T>>r$) and $vec$
is the vectorization, let
\begin{equation}
\mathbf{H}\left( \mathbf{X }\right) =\left( X ,\mathbf{I}_{T}\right) ^{\prime }, \label{def_H_theta}
\end{equation}
where $\mathbf{I}_T$ is $T \times T $
identity matrix. Let
\begin{equation}
\Omega \left( \mathbf{X }\right) _{T\times T}=\left( \mathbf{I}%
_{T}+XX ^{\prime }\right) . \label{var-cov_TS_multi}
\end{equation}
What is the derivative of
\begin{equation}
\frac{\partial \log \left\vert \Omega \left( \mathbf{X }\right)
\right\vert }{\partial \mathbf{x }}=??
\end{equation}
$|A|$ is the determinant of $A$,
and
\begin{equation}
\frac{\partial tr\left[ \mathbf{H}\left( \mathbf{X }\right) ^{\prime
}\Omega \left( \mathbf{X }\right) ^{-1}\mathbf{H}\left( \mathbf{%
X }\right) \right] }{\partial \mathbf{x }}=??
\end{equation}
where $tr$ is the trace operator.
Many thanks!
Best Answer
For typing convenience, let me substitute Latin in place of your Greek letters $$\eqalign{ X &= \Xi \cr x &= \vartheta = \operatorname{vec}(X) \cr W &= \Omega = I + XX^T = W^T \cr\cr }$$
Then for your first function, the differential and gradient can be calculated as $$\eqalign{ f &= \log\det W \cr \cr df &= d\log\det W = d\operatorname{tr}\log W \cr &= W^{-T}:dW \cr &= W^{-T}:(dX\,X^T+X\,dX^T) \cr &= \big(W^{-T}X+W^{-1}X\big):dX \cr &= 2\,W^{-1}X:dX \cr &= 2\,\operatorname{vec}(W^{-1}X)\cdot\,dx \cr \cr \frac{\partial f}{\partial x} &= 2\,\operatorname{vec}(W^{-1}X) \cr\cr }$$ In your second function, I don't quite understand the definition of $\,H(\vartheta)\,$ so all I can offer is a partial solution $$\eqalign{ M &= W^{-1} = M^T \cr f &= H:MH \cr \cr df &= dH:MH + H:M\,dH + H:dM\,H \cr &= (M+M^T)H:dH + HH^T:dM \cr &= 2\,MH:dH - HH^T:M\,dW\,M \cr &= 2\,MH:dH + MHH^TM:dW \cr &= 2\,MH:dH + MHH^TM:(dX\,X^T+X\,dX^T) \cr &= 2\,MH:dH + 2\,MHH^TMX:dX \cr \cr }$$ You can finish off the solution by expanding $dH$ in terms of $dX$, then vectorizing.
In the above, a colon denotes the double-dot (aka Frobenius) product, which is merely a product notation for the trace, i.e. $$A:B=\operatorname{tr}(A^TB)$$
Update
I'm guessing that $H$ is a partitioned matrix: $H=[\,X, I\,]$
Expanding that term in the differential $$\eqalign{ MH:dH &= [\,MX, M\,]:[\,dX, 0\,] \cr &= MX:dX + M:0 \cr }$$ So, continuing with the full differential $$\eqalign{ df &= 2\,\Big(MX + MHH^TMX\Big):dX \cr &= 2\,\operatorname{vec}(MX + MHH^TMX)\cdot dx \cr \cr \frac{\partial f}{\partial x} &= 2\,\operatorname{vec}(MX + MHH^TMX) \cr }$$