How to find a vector derivative with respect to $x\in \mathbb{R}^n$ of
\begin{align}f(x)= (A+B \operatorname{diag}(x))^{-1} b
\end{align}
where $\operatorname{diag}(x)$ is a diagonal matrix where $x$ is a main diagonal, $A\in \mathbb{R}^{n \times n}$, $B\in \mathbb{R}^{n \times n}$, $b \in \mathbb{R}^n$.
This question is similar to what I have asked here. However, there are some differences with matrix multiplication that lead to some confusion for me.
I am also wondering if this can be shown using $\epsilon$-definition of the derivative.
Best Answer
You can obtain the derivative by the chain rule. Let \begin{equation} \begin{array}{ll} \cr\Phi\colon &GL_n({\mathbb R})\to {\mathbb R}^{n\times n}\cr &U \mapsto U^{-1} \end{array} \end{equation} \begin{equation} \begin{array}{l}\cr g \colon &{\mathbb R}^n\to {\mathbb R}^{n\times n}\cr &x \mapsto A + B \operatorname{diag}(x) \end{array} \end{equation} Then $\Phi'(U)\cdot H = -U^{-1} H U^{-1}$ and $g'(x).h = B \operatorname{diag}(h)$, hence by the chain rule \begin{equation} f'(x)\cdot h =- ( A + B \operatorname{diag}(x))^{-1} B \operatorname{diag}(h)( A + B \operatorname{diag}(x))^{-1} b \end{equation} In terms of partial derivatives, it means that \begin{equation} \frac{\partial f}{\partial x_i} =- ( A + B \operatorname{diag}(x))^{-1} B E_{i,i}( A + B \operatorname{diag}(x))^{-1} b \end{equation} where $E_{i, i}$ is the matrix of which all terms are zero but the term of at position $(i, i)$ which value is $1$, or equivalently $E_{i, i} = e_i e_i^T$ where $e_i$ is the i-th basis column vector.
In particular, one sees that $e_i^T ( A + B \operatorname{diag}(x))^{-1} b$ is a scalar, the $i$-th component of $f(x)$ and it follows easily that the Jacobian matrix of $f$, which columns are the vectors $\frac{\partial f}{\partial x_i}$ must be \begin{equation} \partial f = - ( A + B \operatorname{diag}(x))^{-1} B \operatorname{diag}(f(x)) \end{equation}