I was trying to prove the following:
Let $U\Sigma V$ be the $SVD$ decomposition of $A\in\mathbb R^{m\times n}$, where $\textrm{rank}(A)=k$. Show that the pseudoinverse of $A$ is given by,
$$
\displaystyle A^\dagger=\sum_{i=1}^k\sigma_i^{-1}v_iu_i^T.
$$
${\bf Proof:}$ Let us show that $AA^\dagger A=A$, $A^\dagger AA^\dagger=A^\dagger$, $(A^\dagger A)^T=A^\dagger A$ and $(AA^\dagger)^T=AA^\dagger$. Particionate $A$ as follows,
\begin{align*}
\displaystyle A=\left[\begin{array}{c|c}
U_1&U_2
\end{array}\right]\left[\begin{array}{c|c}
\tilde{\Sigma}&0\\ \hline 0&0
\end{array}\right]\left[\begin{array}{c}
V_1^T\\ V_2^T
\end{array}\right],
\end{align*}
where $\tilde{\Sigma}=\textrm{diag}(\sigma_1, \ldots, \sigma_k)$. Then, $A=U_1\tilde{\Sigma}V_1^T$ whereas $A^\dagger=V_1\tilde{\Sigma}^{-1}U_1^T$. Since $V^TV=U^TU=I$ we see $U_1^TU_1=V_1^TV_1=I$. Then,
\begin{align*}
\displaystyle (A^\dagger A)^T=(V_1\tilde{\Sigma}U_1^T)(U_1\tilde{\Sigma}^{-1}V_1^T)=I=A^\dagger A.
\end{align*}
Analogously, we see $(AA^\dagger)=I=AA^\dagger$. Using that $A^\dagger A=AA^\dagger=I$ (how can that be? $A$ is not squared) we have,
\begin{align*}
\displaystyle (A^\dagger AA^\dagger)=(A^\dagger A)A^\dagger=IA^\dagger\ \textrm{and}\ (AA^\dagger A)=A(A^\dagger A)=AI=A.
\end{align*}
${\bf Problems:}$ The main problem is the part $(A^\dagger A)=I$ for $A$ is not a squared matrix. Furthermore if that identity held the problem would be almost trivial.. What is the problem with my proof?
Best Answer
First I guess your SVD decomposition is $A=U\Sigma V^T$ and not $A=U\Sigma V$.
Be carefull of the format of your matrices!
It is true that $V^TV=I_n$ and $U^TU=I_m$ but (unless $n=m$) $I_n\neq I_m$.
Still it is enough to prove that $U_1^T U_1=I_k=V_1^T V_1$.
But then $$\begin{align} A^\dagger\,A & =V_1\tilde \Sigma^{-1} U_1^T U_1\tilde \Sigma V_1^T\\ &=V_1\tilde \Sigma^{-1} I_k\tilde \Sigma V_1^T\\ &=V_1\tilde \Sigma^{-1} \tilde \Sigma V_1^T\\ &=V_1 I_k V_1^T\\ &=V_1 V_1^T\\ &=\left[\begin{array}{c|c} I_k&0\\ \hline 0&0 \end{array}\right] \end{align}\,.$$
And so on.