I am trying to differentiate over the following Frobenius Norm: $$\Phi =||A-(B\circ C)D ||^2_F$$
with respect to B, C, D respectively, i.e.:
$$\frac{\partial \Phi}{\partial B}, \frac{\partial \Phi}{\partial C},\frac{\partial \Phi}{\partial D}$$
After expending the norm and writing it in the trace form, it becomes:
$$\Phi = tr(A^TA) -tr(A^T(B\circ C)D )-tr(D^T(B^T\circ C^T)A)+tr(D^T(B^T \circ C^T)(B \circ C)D) $$
(please correct me if I did the expansion wrong)
I've tried to write them elements by elements and checked out the whole matrix cookbook.
Since there's a hadamard product involving in the equation, I got completely confused of differentiating the last three terms.
Could anyone help me out please? I appreciate a lot.
Best Answer
Define the matrices $$\eqalign{ X &= B\circ C,\quad Y=XD-A \cr }$$ Write the function in terms of these new variables. Then find its differentials and gradients (in alphabetical order). $$\eqalign{ \phi &= \|Y\|_F^2 = Y:Y \cr d\phi &= 2Y:dY \cr &= 2Y:dX\,D \cr &= 2YD^T:C\circ dB \cr &= 2C\circ(YD^T):dB \cr \frac{\partial\phi}{\partial B} &= 2C\circ(YD^T) \cr }$$ Since $B$ and $C$ are interchangeable $$\eqalign{ \frac{\partial\phi}{\partial C} &= 2B\circ(YD^T) \cr }$$ Finally, the gradient wrt $D$ $$\eqalign{ d\phi &= 2Y:X\,dD \cr &= 2X^TY:dD \cr \frac{\partial\phi}{\partial D} &= 2X^TY \cr }$$ NB: In several of the steps above, a colon is used as a convenient product notation for the trace, i.e. $$B:C = {\rm Tr}(B^TC)$$