Assume that we have a positive definite matrix $C$, and a positive definite diagonal matrix $\Lambda$.
Are all the diagonal entries of $(C + \Lambda)^{-1}$ smaller than those of $C^{-1}$? In other words, if we increase the diagonal entries of a positive definite matrix, will the diagonal entries of its inverse decrease?
If so, how to prove it?
Thanks a lot
Best Answer
The proof is as follows:
Consider $C^{-1}-(C+\Lambda)^{-1} $ $$ C^{-1}-(C+\Lambda)^{-1} = C^{-1} \left( [C+\Lambda] - C\right) (C+\Lambda)^{-1} =C^{-1} \Lambda (C+\Lambda)^{-1} = C^{-1} \Lambda (I+C^{-1}\Lambda)^{-1} C^{-1}= C^{-1} (\Lambda^{-1}+C^{-1})^{-1} C^{-1} $$ The last matrix is clearly positive definite. Hence, its diagonals must be positive. This shows that $$\left[C^{-1}\right]_{ii} >\left[(C+\Lambda)^{-1}\right]_{ii}$$
Added in response to OP's question
If $\Lambda$ is only positive semi definite, then $\Lambda + \epsilon I$ is positive definite for any $\epsilon >0$. Hence, by virtue of previous argument $$\left[C^{-1}\right]_{ii} >\left[(C+\Lambda+\epsilon I)^{-1}\right]_{ii}, \forall \epsilon > 0$$ By continuity argument, as $\epsilon \to 0$ $$\left[C^{-1}\right]_{ii} \ge \left[(C+\Lambda)^{-1}\right]_{ii}$$
Note that $\Lambda + \epsilon I$ is positive definite for any $\epsilon >0$ because $$ x^T (\Lambda + \epsilon I) x = x^T \Lambda x + \epsilon x^T x \ge \epsilon x^T x \gt 0 \text{ if $x \neq 0$}$$
The trick of adding $\epsilon I$ to a semi-definite matrix to make is positive definite and then taking the limit as $\epsilon \to 0^+$ is the standard way to extend results from definite to semidefinite.