[Math] Moore-Penrose Inverse and Standard Inverse

inversematrices

I have read that the Moore-Penrose inverse $A^+$ of a matrix $A$ is the same as the standard inverse $A^{-1}$ in the case $A$ is a square, invertible matrix. Is there any relation between the Moore-Penrose inverse and the standard inverse in the case $A$ is a non-invertible square matrix?

The reason I am asking is due to implementing an inverse for a large matrix in R. When I tried the standard inverse, I found out the matrix was non-invertible. But when I tried to find the Moore-Penrose inverse, I got a new matrix.

Best Answer

The Moore-Penrose pseudoinverse $A^+$ has many brilliant properties. First of all, it coincides with the standard inverse $A^{-1}$ when $A^{-1}$ exists. I found, in particular, interesting the use of $A^+$ in relation with systems of linear equations $A\mathbf x=\mathbf b$:

  1. If $A$ is $n\times n$ and invertible, the unique solution is $\mathbf x=A^{-1}\mathbf b=A^{+}\mathbf b$. The following case is more interesting.
  2. If $A$ is non invertible (because $|A|=0$ or $A$ is rectangular and, hence $A^{-1}$ does not exists) then the system can have no or infinite solutions.

No solution: the vector $A^{+}\mathbf b$ is the closest approximation to a solution in least squares sense. In other words, the sum of squared deviations from $b$ is minimal or $||A\mathbf x-b||_2$ is minimal when $\mathbf x=A^{+}\mathbf b$.

Infinite number of solutions: the vector $A^{+}\mathbf b$ is, among all solutions, the one with the shortest euclidean norm (i.e., it is as close to the origin as it is possible)

To sum up, using the pseudoinverse gives the same solution when this is unique. When there is no solution, it still can be used to obtain a reasonable approximation of the solution. Finally, when many solutions are available (underdetermined system), the one produced is the "smallest". Hope this helps.