Is Frobenius-norm projection Lipschitz continuous under operator norm

linear algebramatrix-normsperturbation-theoryprojectionsymmetric matrices

Definition

Let $d \in \mathbb{N}$. Let $A \in \mathbb{R}^{d\times d}$ be a PSD matrix. Define the following operator: $T:\mathbb{R}_{+} \times \mathbb{R}^{d\times d} \to \mathbb{R}^{d\times d} $: Let $\{u_1,\dots,u_d\} \subset \mathbb{R}^d$ and $\{\lambda_1,\dots,\lambda_d\}\subset \mathbb{R}$ be the eigenvectors and eigenvalues of $A$. Then,
\begin{equation}
T_{\lambda_0}(A) = \sum_{i=1}^{d} \max\{\lambda_0,\lambda_i\} u_i u_i^T.
\end{equation}

It is not difficult to see that:

\begin{equation}
T_{\lambda_0}(A) = \arg\min_{\hat{A}\in \mathcal{S}_{+}^{d} : ~\forall x \in \mathbb{R}^d ~ x^T \hat{A} x \ge \lambda_0 \|x\|_2^2}\|\hat{A}-A\|_F.
\end{equation}

where $\|\cdot\|_F$ is Frobenius norm.

In other words, $T_{\lambda_0}(A)$ is the Frobenius-norm projection of $A$ onto the the set of PSD matrices with minimum eigenvalue of $\lambda_0$.

Conjecture

For every $\lambda_0>0$, $A \in \mathbb{R}^{d \times d}$, and $B \in \mathbb{R}^{d \times d}$ such that $A$ and $B$ are PSD, we have
\begin{equation}
\| T_{\lambda_0}(A+B) – T_{\lambda_0}(A) \| \leq \| B \|.
\end{equation}

where $\|\cdot\|$ denotes the operator norm $\ell_2/ \ell_2$. (Notice that we don't use Frobenius norm here.)

I numerically verified this conjecture over the random draws of $A$ and $B$. However, I don't have any proof. Please let me know if you have any hints!

Best Answer

Your conjecture is false. Here is a random counterexample: $$ A=\pmatrix{15&2&5\\ 2&4&6\\ 5&6&12}, \ B=\pmatrix{7\\ &6\\ &&1},\ \lambda_0=8. $$ The LHS of your inequality is $7.0141$ (four decimal places) but the RHS is $\|B\|=7$.

However, the map $T_{\lambda_0}$ is indeed Lipschitz-continuous, although the modulus of continuity is not equal to $1$ but a constant that depends on the size of the matrices. Denote by $|X|$ the matrix absolute value $(X^\ast X)^{1/2}$ of $X$, then $T_{\lambda_0}(A)=\frac12\left(|A-\lambda_0I|+A+\lambda_0I\right)$ and $$ \|T_{\lambda_0}(A+B)-T_{\lambda_0}(A)\| =\frac12\|\,|A+B-\lambda_0I|-|A-\lambda_0I|+B\,\|. $$ By Araki and Yamagami, An Inequality for Hilbert-Schmidt Norm, Commun. Math. Phys. 81: 89-96 (1981), the following inequality holds for any two matrices $X,Y\in M_d(\mathbb C)$: $$ \|\,|X|-|Y|\,\|_F\le\sqrt{2}\|X-Y\|_F. $$ (The authors mentioned that the Lipschitz constant $\sqrt{2}$ can be improved to $1$ when $X$ and $Y$ are self-adjoint, but the proof was given in a paper that I have no access to.) It follows that $$ \|\,|X|-|Y|\,\|\le\|\,|X|-|Y|\,\|_F\le\sqrt{2}\|X-Y\|_F\le\sqrt{2d}\,\|X-Y\|. $$ So, in your case, $$ \begin{align} &\|T_{\lambda_0}(A+B)-T_{\lambda_0}(A)\|\\ &=\frac12\|\,|A+B-\lambda_0I|-|A-\lambda_0I|+B\,\|\\ &\le\frac12\|\,|A+B-\lambda_0I|-|A-\lambda_0I|\,\|+\frac12\|B\|\tag{$\#$}\\ &\le\frac12(\sqrt{2d}+1)\|B\|.\\ \end{align} $$ For more information about the Lipschitz constant for $X\mapsto|X|$, see also R. Bhatia, Modulus of continuity of the matrix absolute value, Indian Journal of Pure and Applied Mathematics volume 41, pages 99–111 (2010).

It is also possible to remove the dependence on $d$ if you drop Lipschitz continuity for continuity. According to Tosio Kato, Continuity of the Map $S\to|S|$ for Linear Operators, Proc. Japan Acad., 49: 157-160 (1973), the map $S\mapsto|S|$ is almost Lipschitz-continuous on the space of all real symmetric matrices, in the sense that $$ \|\,|S|-|T|\,\|\le\frac{2}{\pi}\|S-T\|\left(2+\log\frac{\|S\|+\|T\|}{\|S-T\|}\right). $$ In your case, if we continue from $(\#)$, this translates to $$ \begin{aligned} &\|T_{\lambda_0}(A+B)-T_{\lambda_0}(A)\|\\ &\le\|B\|\left(\frac{2}{\pi}+\frac{1}{2}+\frac{1}{\pi}\log\frac{\|A+B-\lambda_0I\|+\|A-\lambda_0I\|}{\|B\|}\right). \end{aligned} $$

Related Question