Given matrices $\textbf{A}, \textbf{B}, \textbf{C}$ and column vector $\textbf{v}$, what is the derivative of $\langle \textbf{A} \text{diag}(\textbf{B}\textbf{v}), \textbf{C} \rangle$ with respect to $\textbf{B}$? $\text{diag}(\cdot)$ is a diagonal matrix with the argument as the diagonal and the brackets signify the inner product.
I am trying to find the solution, but am hung up on differentiating the diagonal matrix.
Any assistance would be greatly appreciated. Thank you very much.
Best Answer
The derivative can always be computed by going back to the original limit definition. For a vector-valued function $f: X \rightarrow Y$, its derivative at a point $x$, denoted $Df(x)$, if it exists, is given by $$[Df(x)]z = \lim_{h \rightarrow 0} h^{-1} (f(x + hz) - f(x)).$$ In other words, if the true (Frechet) derivative exists, then you can compute it by computing directional (Gateaux) derivatives.
In your case, $f(X) = \langle A \mathrm{diag}(Xv), C \rangle$, where $X$ is the variable ($B$ in your original question). Then perhaps you can verify that $$[Df(X)]Z = \langle A \mathrm{diag}(Zv), C \rangle$$.