[Math] L2 Norm of Pseudo-Inverse Relation with Minimum Singular Value

linear algebramatricesnormed-spacespseudoinverse

Consider a matrix $A \in\mathbb R^{n\times m}$ with $n>m$. It has full column rank, i.e. $\operatorname{rank}(A)=m$. Its left pseudo-inverse is given by; $$A^{-1}_\text{left}=(A^TA)^{-1}A^T $$

From two different results during my studies, I have realized the following:
$$ \|A^{-1}_\text{left}\|_2 = \frac{1}{\sigma_{\min}(A)} $$
just like the case as if $A$ is square invertible matrix.

I have seen a similar question, however I couldn't relate the answer with the equality given above.

My question is: How can we show that the L2 norm of left pseudo-inverse of $A$ is related to its minimum singular value?

Thank you in advance for your help.

Best Answer

Hint: It suffices to prove the following facts:

  • If $U,V$ are orthogonal (and square), then $\|AV\| = \|UA\| = \|A\|$
  • $(U\Sigma V^T)^\dagger = V\Sigma^{\dagger}U^T$ (where $\dagger$ denotes the pseudo-inverse)
  • If $\Sigma$ is a diagonal matrix of singular values, then $\|\Sigma^{\dagger}\| = 1/\sigma_{min}(\Sigma)$

Perhaps you can put the pieces together from here. Happily, this approach still works when $A$ does not have full column-rank.

Related Question