For the function
$f({\bf x}) = ||{\bf Ax – b}||^2_2$, where the vectors are ${\bf x} \in \mathbb{C}^{n \times 1}, {\bf y} \in \mathbb{C}^{m \times 1}$ and martix ${\bf A} \in \mathbb{C}^{m \times n}$, which has a Lipschitz continus gradient, I can write the Lipschitz smoothness condition as:
$f({\bf y}) \leq f({\bf x}) + 2 {\rm Re}\{\nabla f({\bf x})^H ({\bf y} – {\bf x}) \}+ ({\bf y}-{\bf x})^H \nabla^2 f({\bf x}) ({\bf y}-{\bf x}) \leq f({\bf x}) + 2 {\rm Re}\{\nabla f({\bf x})^H ({\bf y} – {\bf x})\} + \frac{L}{2}||{\bf y}-{\bf x}||_2^2$,
where L is Lipschitz constant.
I have the following function. Now, I am looking for the Lipschitz condition for the following Frobenius norm funtion:
$f({\bf B}) = ||{\bf A} – {\bf BC}||^2_F$, where ${\bf A} \in \mathbb{C}^{m \times n}$, ${\bf B} \in \mathbb{C}^{m \times k}$, and ${\bf C} \in \mathbb{C}^{k \times n}$, which is a convex smooth function w.r.t ${\bf B}$.
Based on my previous two posts:
Taylor expansion of Frobenius Norm
Second order Taylor expansion of Frobenius norm
For a given $\bar{\bf B}$, I have the following solution for it:
$||{\bf A – {\bf BC}}||^2_F \leq ||{{ {\bf A}- \bar{\bf B} {\bf C}}}||^2_F + 2 {\rm Re} {\rm Tr} \left\{\nabla{{\bf g}}\left(\bar{\bf B}\right)^H \left({\bf B} – \bar{\bf B} \right) \right\} + \frac{L}{2} ||{{\bf B} – \bar{\bf B}}||_F^2,$
where $\nabla{{\bf g}}\left(\bar{\bf B}\right) = (\bar{\bf B}{\bf C – A}){\bf C}^H$ and $L = ||{\bf C}{\bf C}^H||^2_F$.
Can someone kindly suggest if this is correct or I am doing some mistake?
Best Answer
You have $B=\bar B+X$. Then $$ \|A−BC\|_F^2=\|A−\bar BC-XC\|_F^2=\|R-XC\|_F^2=\|R\|_F^2-2Re\langle R,XC\rangle_F+\|XC\|_F^2 $$ And indeed $$ ⟨R,XC⟩_F=Tr(R^HXC)=Tr(CR^HX)=⟨RC^H,X⟩_F $$ but $$ \|XC\|_F=\sqrt{\sum_{k,l} |e_k^HXCe_l|^2}\le\sqrt{\sum_{k} \|e_k^HX\|_2^2\sum_l\|Ce_l\|_2^2}=\|X\|_F\|C\|_F $$ so that $L=2\|C\|_F^2$.