Gradient with respect to x

calculusderivativeslinear algebramatricesvector analysis

What is the gradient (with respect to $x \in \mathbb{R}^n$) of $f(\vec{x}) = e^{\vec{x}^TA\vec{x}+b}$ where $A \in \mathbb{R}^{n\times n}$ is a symmetric matrix and $b$ is a real number?

I know that the derivative of $\vec{x}^TA\vec{x}+b$ is $2\vec{x}^TA$, but I am not sure of how to get the gradient of $f(\vec{x}) = e^{\vec{x}^TA\vec{x}+b}$ with respect to $x$?

Best Answer

This is just the chain rule.

Let:

  • $\alpha: \mathbb{R}^n \rightarrow \mathbb{R}$ such that $\alpha(x)=xAx+b$
  • $\exp: \mathbb{R} \rightarrow \mathbb{R} $ such that $exp(t)=e^t$

Then you have that

$$ f(x)=(exp \, \circ \, \alpha)(x) $$

thus, buy chain rule for the gradient

$$ \nabla f (x) = \nabla exp(\alpha(x)) \, \cdot \, \nabla \alpha (x) $$

$exp$ being from the reals to the reals, the first gradient is simply a derivative: $ \nabla exp(t) = \frac{d}{dt}|_t e^t = e^t $, while $ \nabla \alpha (x) = \nabla (xAx+b)(x) = \nabla (xAx) +\nabla b = \nabla xAx = 2Ax $ (the latest term should be proven but you can also take it for granted).

In the end:

$$ \nabla f (x)= 2e^{xAx+b} (A\cdot x)$$ where $ 2e^{xAx+b} $ is a number and $ (A\cdot x) $ is a vector.

Related Question