What is the gradient (with respect to $x \in \mathbb{R}^n$) of $f(\vec{x}) = e^{\vec{x}^TA\vec{x}+b}$ where $A \in \mathbb{R}^{n\times n}$ is a symmetric matrix and $b$ is a real number?
I know that the derivative of $\vec{x}^TA\vec{x}+b$ is $2\vec{x}^TA$, but I am not sure of how to get the gradient of $f(\vec{x}) = e^{\vec{x}^TA\vec{x}+b}$ with respect to $x$?
Best Answer
This is just the chain rule.
Let:
Then you have that
$$ f(x)=(exp \, \circ \, \alpha)(x) $$
thus, buy chain rule for the gradient
$$ \nabla f (x) = \nabla exp(\alpha(x)) \, \cdot \, \nabla \alpha (x) $$
$exp$ being from the reals to the reals, the first gradient is simply a derivative: $ \nabla exp(t) = \frac{d}{dt}|_t e^t = e^t $, while $ \nabla \alpha (x) = \nabla (xAx+b)(x) = \nabla (xAx) +\nabla b = \nabla xAx = 2Ax $ (the latest term should be proven but you can also take it for granted).
In the end:
$$ \nabla f (x)= 2e^{xAx+b} (A\cdot x)$$ where $ 2e^{xAx+b} $ is a number and $ (A\cdot x) $ is a vector.