I have a matrix that ought to be symmetric and positive semi-definite (with real entries). However, due to small rounding errors that can accumulate over time, that might not be strictly true. For example, the smallest eigenvalue might be $-10^{-10}$ even though it ought to be $0$ if we had been using exact calculations. This isn't particularly important overall, except that some computations will fail altogether with even this small negative number.
Is there a simple expression that will fix this? As an example of what I'm looking for:
- If I were asking "how do I make a real number non-negative if it already nearly is?" I'd like the answer "take the absolute value". This leaves non-negative numbers precisely unchanged, and only has a small effect on small strictly negative numbers.
- If I were asking "how do I make a matrix $M$ symmetric" I'd like the answer "use $M + M^t$". Actually that is one of the things I want so that will be my first step!
- So what I'm asking, analogously, is: "how do I make the matrix positive semidefinite?"
Best Answer
One possible method is to take the operator $$ |M| = (M^T M)^{1/2}. $$ This is a positive semidefinite matrix whose eigenvalues are the singular values of $M$. For example, if $M$ is a symmetric matrix then $|M|$ is a symmetric matrix matrix whose eigenvalues are the absolute value of the eigenvalues of $M$. Indeed, if $M$ is symmetric then let $M = U D U^T$ be the spectral decomposition of $M$ where $U$ is an orthogonal matrix and $D$ is diagonal. Then $$ |M| = ((UDU^T)^T (UDU^T))^{1/2} = (UDU^T U DU^T)^{1/2} = U(D^2)^{1/2}U^T = U |D| U^T. $$ It also follows then that if $M$ is already positive semidefinite then $|M| = M$.
If you want this to work in general for matrices acting on $\mathbb{C}^n$ then you can also take $|M| = (M^* M)^{1/2}$ where $~^*$ indicates the conjugate transpose.