Moore-Penrose Inverse of $A^{T}A$

matricespseudoinverse

The Moore-Penrose inverse of a rectangular matrix is given by:
$$
A^{+}=V_{r}S_{r}^{-1}U_{r}^{T}
$$
Here I am using the economical size SVD. Where $U_{r}^{T}U_{r}=V_{r}^{T}V_{r}=I_{r}$. My question is

It is known that $(A^{T}A)^{+}=V_{r}S_{r}^{-2}V_{r}^{T}$ so my question is how can we show it instead of proving $(A^{T}A)^{+}(A^{T}A)=I_{r}$?

Edit :

I would like to thank user @cangrejo for his suggesstion which I find useful as it confirms my assumption that this problem can be tackled by using properties of the pseudo-inverse :
$$
(A^{T}A)^{+}=A^{+}(A^{+})^{T}
$$
However, the wikipedia article en.wikipedia.org/wiki/Moore%E2%80%93Penrose_inverse under the section properties lists one of the sufficient conditions for $(AB)^{+}=B^{+}A^{+}$ is that $B=A^{T}$ which is similar to my case but how can we prove it to be a sufficient condition to the main result $(AB)^{+}=B^{+}A^{+}$ for which this would help me complete the main objective of finding pseudoinverse of $A^{T}A$? In other words, how do we show that if the expression $B=A^{T}$ is true, then this will allow me to write $(AB)^{+}=B^{+}A^{+}$

I will award the bounty for a complete answer.

Best Answer

A general strategy for proving statements about the Moore-Penrose inverse is to (1) use a SVD to reduce statements about general matrices to statements about diagonal matrices, and (2) use the 4 Moore-Penrose conditions:

  1. $X Y X = X$
  2. $Y X Y = Y$
  3. $(XY)^T = XY$
  4. $(YX)^T = Y X$

If $Y$ satisfies these 4 equations, then $Y=X^+$. Note that the pseudo-inverse is always unique. So, all we need to do to prove $B=A^T\implies (AB)^+ = B^+ A^+$ is to verify that $Y=B^+A^+$ satisfies these 4 equations for $X=AB$.

Given the economical SVD $A=USV^T$, and $B=A^T = V^T S^T U = V^T S U$ we have

  • $A^+ = VS^{-1}U^T$
  • $B^+ = US^{-1}V^T$
  • $AB = (USV^T)(VSU^T) = US^2 U^T$
  • $B^+A^+ = (US^{-1}V^T)(VS^{-1}U^T) = US^{-2}U^T$

Plugging these into the 4 MP conditions yields straightforward computations

  1. $(AB)(B^+A^+)(AB) = \big(US^2U^T\big)\big(U(S^{-2}U^T\big)\big(US^2U^T\big) = US^2 U^T = AB$

  2. $(B^+A^+)(AB)(B^+A^+) = \big(U(S^{-2}U^T\big)\big(US^2U^T\big)\big(U(S^{-2}U^T\big) = US^{-2} U^T = B^+A^+$

  3. $\big((AB)(B^+A^+)\big)^T = \Big(\big(US^2U^T\big)\big(U(S^{-2}U^T\big)\Big)^T = (UU^T)^T = UU^T= (AB)(B^+A^+)$

  4. $\big((B^+A^+)(AB)\big)^T = \Big(\big(U(S^{-2}U^T\big)\big(US^2U^T\big)\Big)^T = (UU^T)^T = UU^T= (B^+A^+)(AB)$

Notice how, due to the cancellation of the (semi-)orthogonal matrices, all statements are effectively reduced to statements about diagonal matrices.

Related Question