Given any real-valued, symmetric, positive semi-definite matrix $A, B \in \mathbb{R}^{n \times n}$, $\lambda > 0$. Does the following inequality hold for some constant $c$?
$$\sigma_{\max}\Big((A+B + \lambda I_n)^{-1}A\Big) \leq c,$$
where $\sigma_{\max}(\cdot)$ denotes the maximal singular value.
The constant $c$ might be dependent on $\lambda$ and some norm of $A$ or $B$.
Best Answer
No. Let $A_0=\pmatrix{1&0\\ 0&0}, B_0=\pmatrix{1&1\\ 1&1}$ and $$ C_0=(A_0+B_0)^{-1}A_0=\pmatrix{1&-1\\ -1&2}\pmatrix{1&0\\ 0&0} =\pmatrix{1&0\\ -1&0}. $$ Then $\sigma_1(C_0)=\|C_0\|_2>1$ because the Euclidean norm of the first column of $C_0$ is greater than $1$.
It follows that if $A=A_0+tI$ and $B=B_0+\lambda I$ for some small $t,\lambda>0$, then $A$ is positive definite, $B\succeq\lambda I$ and $\|C\|_2>1$ for $C=(A+B)^{-1}A$.
Similarly, if $A=X_1X_1^T+\cdots+X_pX_p^T$ and $X_{p+1}X_{p+1}^T+\cdots+X_tX_t^T$ are close to $A_0$ and $B_0$ respectively and $\lambda>0$ is small, then $\|C\|_2>1$ when $C=(A+B+\lambda I)^{-1}A$.