[Math] Derivative of the inverse of a symmetric matrix

matrix-calculus

I am quite a beginner in linear algebra and matrix calculus. I was wondering what is the derivative of the matrix inverse when the matrix is symmetric. More precisely, I'm looking for $\frac{\partial}{\partial \mathbf{X}} \mathbf{X}^{-1}$ when $\mathbf{X}$ is a symmetric matrix.

I am asking this because I have a function $f: \mathbb{R}^{n\times n} \to \mathbb{R}$ in the form of
\begin{equation*}
f(\mathbf{X}) = \mathrm{trace}(\mathbf{A} \mathbf{X}^{-1}) – \log |\mathbf{X}|
\end{equation*}
and I want to find its extremums using derivatives. I also know if $\mathbf{X}$ is symmetric, then
\begin{align*}
\frac{\partial \mathrm{trace} (\mathbf{A} \mathbf{X})}{\partial \mathbf{X}} & = \mathbf{A} + \mathbf{A}^T – (\mathbf{A} \circ \mathbf{I}) \\
\frac{\partial \log |\mathbf{X}|}{\partial \mathbf{X}} & = 2 \mathbf{X}^{-1} – (\mathbf{X}^{-1} \circ \mathbf{I})
\end{align*}
(from http://www.mit.edu/~wingated/stuff_i_use/matrix_cookbook.pdf Section 2.5).

I somehow want to use the above with the chain rule to write
\begin{equation*}
\frac{\partial f}{\partial \mathbf{X}} = \frac{\partial f}{\partial \mathbf{X}^{-1}} \frac{\partial \mathbf{X}^{-1}}{\partial \mathbf{X}}
\end{equation*}
and compute the derivative of $f$ with respect to $\mathbf{X}$ (since I can easily write $\log |\mathbf{X}| = – \log |\mathbf{X}^{-1}|$).

Best Answer

Hint: use $(X^{-1})_{ij}=\frac{C_{ji}}{\operatorname{det}(X)}$, with $C_{ji}=(-1)^{i+j}X_{ji}=(-1)^{i+j}X_{ij}$, as $X$ is symmetric.

Then

$$\frac{\partial (X^{-1})_{ij}}{\partial X_{kl}}= \frac{\partial }{\partial X_{kl}}\left( \frac{(-1)^{i+j}X_{ij}}{\operatorname{det}(X)}\right).$$

Using the formula for the derivative of the determinant of $X$

$$\frac{\partial \det(X)}{\partial X_{kl}}= \det(X)(X^{-1})_{lk}$$

you can arrive at the result.