Let $A$ be an $n\times n$ real invertible matrix, and $\delta A$ be a $n\times n$ matrix such that $A+\delta A$ is invertible. Then, it is known that
$$
\frac{\left\|(A+\delta A)^{-1} – A^{-1}\right\|}{\left\|A^{-1}\right\|} \leq \kappa(A)\frac{\left\|\delta A\right\|}{\left\|A\right\|} + O\left(\left\|\delta A\right\|^2\right), \tag{1}
$$
where $\|\cdot \|$ is the $2$-norm (i.e., the spectral norm) and $\kappa(A)$ is the associated condition number.
My question is: is there a way to obtain a bound sharper than (1) in case $A$ and $\delta A$ are both symmetric positive definite?
Best Answer
For my convenience, I will replace $\delta A$ with $H$. Note that $$ (A + H)^{-1} = (A^{1/2}[I + A^{-1/2}HA^{-1/2}]A^{1/2}) \\ = A^{-1/2}(I + A^{-1/2}HA^{-1/2})A^{-1/2}, $$ so that $$ (A + H)^{-1} - A^{-1} = \\ A^{-1/2}[(I + K)^{-1} - I]A^{-1/2}, $$ where $K = A^{-1/2}HA^{-1/2}$. It follows that $$ \frac{\|(A + H)^{-1} - A^{-1}\|}{\|A^{-1}\|} = \frac{\|A^{-1/2}[(I + K)^{-1} - I]A^{-1/2}\|}{\|A^{-1}\|} \\ \leq \frac{\|A^{-1/2}\|\cdot \|(I + K)^{-1} - I\| \cdot \|A^{-1/2}\|}{\|A^{-1}\|} \\ = \frac{\|A^{-1}\|\cdot \|(I + K)^{-1} - I\|}{\|A^{-1}\|} \\ = \|(I + K)^{-1} - I\|. $$ Now, $K = (I + K) - I$ is positive definite. It follows that $I^{-1} - (I + K)^{-1} = K(I + K)^{-1}$ is positive definite. Thus, noting that $x \mapsto x/(1 + x)$ is increasing for $x \geq 0$, we have $$ \|(I + K)^{-1} - I\| = \lambda_{\max}(K(I + K)^{-1}) = \lambda_{\max}(K)(1 + \lambda_{\max}(K))^{-1}. $$ Notably, we have $$ \lambda_{\max}(K) = \|A^{-1/2}HA^{-1/2}\| \leq \|A^{-1/2}\|^2 \|H\| = \|A^{-1}\| \cdot \|H\|. $$ All together, this gets us the inequality $$ \frac{(A + H)^{-1} - A^{-1}}{\|A^{-1}\|} \leq \frac{\|A^{-1}\|\cdot \|H\|}{1 + \|A^{-1}\| \cdot \|H\|} < \|A^{-1}\| \cdot \|H\| = \kappa(A) \frac{\|H\|}{\|A\|}. $$