Some (hopefully) relevant facts (according to the matrix cookbook)
-
$\frac{\partial}{\partial X} \text{Tr}( \mathbb{F}(X)) = f(X)^T$
- where $\mathbb{F}(\cdot)$ is a differentiable function of each of the elements of $X$ and $f(\cdot)$ the scalar derivative of $\mathbb{F}(\cdot)$.
-
$\partial (X \circ Y) = (\partial X) \circ Y + X \circ (\partial Y)$
-
$\frac{\partial}{\partial X} \| X \|_F^2 = \frac{\partial}{\partial X} \text{Tr}(XX^T) = 2X$
I have a fixed $M$ which is a masking matrix – it has $1$ on certain elements, and $0$ elsewhere. I am trying to combine the facts above to get: $$ \frac{\partial}{\partial F} \|F \circ M \|_F^2$$
In the hopes of setting it equal to the $0$ matrix, and having some kind of closed form relationship. How can I find a formula for the quantity above, and is it possible to have a closed form expression?
So far, I've combined the above to get:
$$\frac{\partial}{\partial F}\text{Tr}((F \circ M)(F \circ M)^T) = 2(F\circ M)\cdot((\partial F) \circ M + F \circ (\partial M))$$
But I'm not sure this correct or where to go from here.
If this is too complicated, can't I use the bound in the following question: $$\| A \circ B\|_F \leq \text{Tr}(AB^T) \leq \| A\|_F \| B\|_F$$ and instead of my minimization with my original term, minimize my problem using the bounds provided here?
Best Answer
Replace the trace with the double-dot (aka Frobenius) product $$\operatorname{tr}(A^TB)=A:B$$ in your masked function.
Then finding the differential and gradient is simple $$\eqalign{ L &= (M\circ F):(M\circ F) \cr\cr dL &= 2\,(M\circ F):(M\circ dF) \cr &= 2\,(M\circ M\circ F):dF \cr &= 2\,(M\circ F):dF \cr\cr \frac{\partial L}{\partial F} &= 2\,M\circ F \cr\cr }$$ In the 3rd line, I made use of the fact that Frobenius and Hadamard products are mutually commutative, i.e. $$A\circ B:X = A:B\circ X$$
Setting the gradient to zero, yields little useful info. Elements in $X$ corresponding to the zero elements of $M$ are unbounded, while the elements corresponding to the unity elements of $M$ are zero. $$\eqalign{ \cr }$$