Derivative of hadamard product of

derivativeshadamard-productlinear algebra

I am having difficulties to compute the derivative of the following expression:

$$ (xx^T)\circ A $$

with respect to $x \in R^K $ where $A \in R^{K\times K}$.

Although Derivative of Hadamard product explains derivatives for the Hadamard product of two matrices, I could not extend it to my expression.

Best Answer

We define a function as $f: \mathbb{R}^k \to \mathbb{R}^{k \times k}, f(x)=(xx^T)\circ A$.

The domain of the function are vectors in $\mathbb{R}^k$ and the range of the function are matrices in $\mathbb{R}^{k \times k}$.

We want to evaluate the derivative of the function with respect to $x$. For each entries of the matrix, we can calculate the partial derivative of that entry with respect to the the different entries in the vector $x$. The result can therefore be organized into a 3rd order tensor of dimension $k \times k \times k$ that we can denote by $\frac{\partial f(x)}{\partial x}$ where the partial derivative of the $i,j$ entry of $f(x)$ with respect to $k^{th}$ entry in $x$ is denoted by $\frac{\partial f(x)}{\partial x}_{i,j,k}$. We know that the $i,j$ entry of $f(x)_{i,j}=x_ix_ja_{ij}$, so $$\frac{\partial f(x)}{\partial x}_{ijk}=\begin{cases} a_{ij}x_j, s=i\\ a_{ij}x_i, s=j\\ 0 , \text{ otherwise} \end{cases}$$