Gradient of $x \mapsto \log \left( \det \left( \dfrac {3}{10}I+xx^{T}\right) \right) $

derivativesdeterminantmultivariable-calculusscalar-fields

Let $f : \mathbb{R}^{n} \to \mathbb{R}$ be defined by

$$f (x) := \log \left( \det \left( \dfrac {3}{10}I+xx^{T}\right) \right)$$

Calculate the gradient $\nabla f$.

I have been struggling to calculate $\nabla f$. My attempt is as follows.

$$\dfrac {df}{dx} = \left( \det \left( \dfrac {3}{10}I+xx^{T}\right) \right)^{-1} \cdot\dfrac {dg}{dx}$$

where $m : = \left(\dfrac {3}{10}I+xx^{T}\right)$

$$g := \det(m) = \det \left( \dfrac {3}{10}I+xx^{T}\right)$$

$$ \dfrac {dg}{dx} = \det \left( \dfrac {3}{10}I+xx^{T}\right).Tr( \left( \dfrac {3}{10}I+xx^{T}\right)^{-1}.\dfrac {dm}{dx}) $$

$$\dfrac {dm}{dx} = \dfrac {d(xx^{T})}{dx}$$

is an $n \times n \times n$ tensor, and $Tr(\dfrac {dm}{dx})= 2x$. So, I get

$$\dfrac {df}{dx} = Tr(\left( \dfrac {3}{10}I+xx^{T}\right)^{-1}.\dfrac {d(xx^{T})}{dx})$$

A second question is about checking this or other complex multivariate differentials, I know my attempts are wrong by using autograd in Python. It would be really helpful to know if this is possible with SymPy or something similar, or it is rarely done.

Best Answer

Let $M = \left( xx^T + 0.3I \right)$ and $f = \log \det \left( M \right)$.

We will utilize the following the identities

  • Trace and Frobenius product relation $$A:B={\rm tr}(A^TB)$$ or $$A^T:B={\rm tr}(AB)$$
  • Cyclic property of Trace/Frobenius product $$\eqalign{ A:BC &= AC^T:B \cr &= B^TA:C \cr &= {\text etc.} \cr }$$
  • Jacobi's formula (for nonsingular matrix $M$) in terms of differential $$d\log \det \left( M \right) = d{\rm tr}\log\left( M \right) .$$

Now, we obtain the differential first and thereafter we obtain the gradient.

So, \begin{align} df &= d \log \det \left( M \right) \\ &= d \ {\rm tr}\left( \log\left( M \right) \right) \hspace{8mm} \text{note: utilized Jacobi's formula} \\ &= {\rm tr} \left( M^{-1} dM \right) \\ &= M^{-T} \ : \ dM \hspace{8mm} \text{note: utilized trace and Frobenius relation} \\ &=M^{-1}\ : \ \left( dxx^T + xdx^T\right)\\ &= 2M^{-1}x \ : \ dx \\ \end{align}

So, the derivative of $f = \log \det \left( xx^T + 0.3I \right)$ with respect to $x$ is \begin{align} \frac{\partial}{\partial x} f = \frac{\partial}{\partial x} \log \det \left( xx^T + 0.3I \right) = 2 \left(xx^T + 0.3I\right)^{-1} x .\\ \end{align}

Related Question