If Y is the pseudoinverse of matrix X, then X will be the pseudoinverse of Y. This is a trivial consequence once the Moore-Penrose conditions are written:
$Y = X^+$ implies
$$
\begin{aligned}
XYX&=X\\
YXY&=Y\\
(XY)^T&=XY\\
(YX)^T&=YX
\end{aligned}
$$
Let $Z=Y^+$. This would mean
$$
\begin{aligned}
YZY&=Y\\
ZYZ&=Z\\
(YZ)^T&=YZ\\
(ZY)^T&=ZY
\end{aligned}
$$
Substituting $Z=X$ results in the second set of conditions being identical to the first.
However, when I try to simplify the expression for the Moore-Penrose inverse, the resulting equations are a mess, and I don't see how to move forward.
$$
\begin{align}
Y &= (X^TX)^{-1}X^T\\
\implies Y^T &= X(X^TX)^{-T} = X(X^TX)^{-1}\\
Y^TY &= X(X^TX)^{-1}(X^TX)^{-1}X^T\\
\implies (Y^TY)^{-1}Y^T &= [X(X^TX)^{-1}(X^TX)^{-1}X^T]^{-1}X(X^TX)^{-1}
\end{align}
$$
Any ideas on how that last expression reduces to $X$? Matrix algebra proofs appreciated.
Best Answer
Unless $X$ is a square and nonsingular, it is not equal to $(Y^TY)^{-1}Y^T$.
When you write $Y=(X^TX)^{-1}X^T$, you are assuming that $X$ has full column rank (otherwise $X^TX$ is not invertible). It follows that $X$ is a "tall" matrix, i.e. $X$ is $m\times n$ for some $m\ge n$. Hence $Y$ is a "fat" matrix. So, when $m>n$, $Y$ has deficient column rank and $Y^TY$ cannot possibly be invertible.
The correct expression of $X$ in terms of $Y$ should be $X=Y^T(YY^T)^{-1}$: \begin{aligned} Y^T(YY^T)^{-1} &=\left((X^TX)^{-1}X^T\right)^T\left[(X^TX)^{-1}X^T\left((X^TX)^{-1}X^T\right)^T\right]^{-1}\\ &=X(X^TX)^{-1}\left[(X^TX)^{-1}X^TX(X^TX)^{-1}\right]^{-1}\\ &=X(X^TX)^{-1}\left[(X^TX)^{-1}\right]^{-1}\\ &=X. \end{aligned}