This question is similar to How to take the gradient of the quadratic form?
But I do not like the answer to that question.
The reason is because the answerer is mixing $\nabla$ with $\dfrac{\partial}{\partial x}$. What I have learned is that $\nabla f$ is a column vector, it is the gradient of $f$, and$\dfrac{\partial}{\partial x} f$ is the Jacobian of $f$, which evaluates to a row vector when $f$ is a scalar.
Why is it even appropriate to mix these notations…?
… In any case, is there a straight forward way of using properties of $\nabla$ to show $$\nabla x^TAx = (A + A^T)x, A \in \mathbb{R}^{n \times n}, x \in \mathbb{R}^n$$
I like the approach used in the linked question. Let $y(x) = Ax$, then
$$\nabla x^TAx = \nabla x^Ty(x)$$
Is there a way to use a chain rule for $\nabla$ at this stage?
I'm thinking of something like
$$\nabla x^TAx = \nabla x^Ty(x) = y(x)^T \nabla x + x^T\nabla y(x)$$
But I have no idea if the above holds.
Best Answer
Here is a proof from first principles:
Let $$f(\mathbf{x})= \mathbf{x}^T A \mathbf{x},$$ and observe that $$f(\mathbf{x}+\mathbf{h})-f(\mathbf{x})=(\mathbf{x}+\mathbf{h})^T A( \mathbf{x}+\mathbf{h})-\mathbf{x}^T A \mathbf{x}=\mathbf{x}^T A \mathbf{h}+\mathbf{h}^T A \mathbf{x}+\mathbf{h}^T A \mathbf{h}. $$ As the last term is of order $\mathcal{O} \left(|\mathbf{h}|^2\right)$, the differential of $f$ is the linear map $$df_\mathbf{x} (\mathbf{h})=\mathbf{x}^T A \mathbf{h}+\mathbf{h}^T A \mathbf{x}=\mathbf{x}^T A \mathbf{h}+\mathbf{x}^T A^T \mathbf{h}=\left( \mathbf{x}^TA+\mathbf{x}^T A^T \right) \mathbf{h} = \left( A^T \mathbf{x}+A \mathbf{x} \right) \mathbf{h}. $$