Derivative of quadratic form of vector-valued function

Question

This seems like a trivial question but I am currently stuck and cannot see what I am doing wrong.

So let us consider a function $f(x) : \mathbb{R}^d \rightarrow \mathbb{R}^d$.

I want to compute the derivative w.r.t. $x \in \mathbb{R}^d$ of an expression that contains a quadratic form of $f(x)$

$$I = f(x)^{\top} C f(x) . $$

Here $C$ is a $d\times d$ matrix.

By taking the derivative w.r.t to the vector $x$ we have

$$ \frac{\partial I}{\partial x} = 2C f(x) \cdot \nabla f(x), $$
where $\nabla f(x)$ denotes the Jacobian of $f$ which will be a $d \times d$ matrix.

Now my problem is that the dimensions of the matrices in the last expression do not match:
We have

• $C: d\times d$,
• $f(x): d\times 1$, and
• $\nabla f(x): d \times d$.

So the last two dimensions do not add up.
What I am doing wrong? Is the correct derivative $$ \frac{\partial I}{\partial x} = \nabla f(x) 2 C f(x) , $$ or $$ \frac{\partial I}{\partial x} = ( 2 C f(x) )^{\top} \cdot \nabla f(x) $$

Answer 1

Let $(\nabla f)_{ik}=\frac{\partial f_i}{\partial x_k}$. Then derivative is

$$ \frac{\partial I}{\partial x} = (\nabla f)^T C f + (\nabla f)^T C^T f. $$

I think you perhaps implicitly assumed that $C$ is symmetric