How to get the derivative of the 2-norm of a vector with respect to a matrix？
For
$$
f(\textbf{X})=\|\textbf{aX}\|^2_2,
$$
it is easy to obtain
$$
\frac{\text{d}f}{d\textbf{X}}=\textbf{a}^T\textbf{a}^*\textbf{X}^*.
$$
where $\textbf{X}\in\mathbb{C}^{l\times l}$, $\textbf{a}\in\mathbb{C}^{1\times a}$.
But for
$$
f(\textbf{X})=\|\textbf{aX}\|_2,
$$
how to get the derivative?
Best Answer
It is actually quite simple. Let $f(\mathbf{X})=g^2(\mathbf{X})$
Using differential, you will find
$$ df = \frac{\partial f}{\partial \mathbf{X}} : d\mathbf{X} = 2 g(\mathbf{X}) dg $$ Here : denotes the Frobenius inner product.
It follows $$ dg =\frac{1}{2 g(\mathbf{X})} \cdot \frac{\partial f}{\partial \mathbf{X}} : d\mathbf{X} $$ The gradient is thus $$ \frac{\partial g}{\partial \mathbf{X}} =\frac{1}{2 g(\mathbf{X})} \cdot \frac{\partial f}{\partial \mathbf{X}} $$