I would like to compute a partial inverse of a symmetric semi-definite matrix.
I read about computing the pseudoinverse of a rectangular matrix by using SVD, however with a symmetric matrix I could apply a similar technique using instead the eigenvalue decomposition, i.e. compute the eigenvalues, discard the smallest and invert the remaining.
Does this make sense? If so can you point me to a reference that explains how to achieve this in detail?
Any help appreciated.
Best Answer
The idea is that the singular value decomposition,
$$\mathbf A=\mathbf U\mathbf \Sigma\mathbf V^\top$$
and the eigendecomposition
$$\mathbf A=\mathbf Q\mathbf D\mathbf Q^\top$$
of a symmetric matrix are one and the same.
Thus, if one wants the Moore-Penrose pseudoinverse of $\mathbf A$, either decomposition could be used. (However, an SVD routine generally wouldn't exploit the nice structure of a symmetric matrix, so a bit more computational effort than what is actually needed will be used in that case; thus, use the eigendecomposition.)
The idea is that, letting $\mathbf A^\dagger$ be the Moore-Penrose pseudoinverse, we have the property
$$\mathbf A^\dagger=\mathbf Q\mathbf D^\dagger\mathbf Q^\top$$
where $\mathbf D^\dagger$ is (usually) computed via the following procedure: take $d_1$ to be the largest eigenvalue, and let $\varepsilon$ be machine epsilon. Reciprocate any entry of $\mathbf D$ that is greater than $\varepsilon\cdot d_1$, and set all other entries to zero.