In customary formulations of the Singular Value Decomposition or SVD that I have seen,
(e.g., Wikipedia or Gil Strang's textbooks) it is always stated in terms of writing an
$m \times n$ matrix $M$ (say of rank $r$) as a product $U \Lambda V$, where $U$ and $V$ are
orthogonal $m \times m$ and $n \times n$ matrices and $\Lambda$ is a diagonal $m \times n$ matrix
with non-negative "singular values" on the diagonal, $s_1 \ge s_2 \ge \ldots \ge s_r >0$ and the
rest zero. Looking back over the many times I have taught linear algebra to both undergraduates
and graduate students, I realized that I have not once covered the SVD, and even though I consider
myself pretty knowledgeable about linear algebra, I have never felt comfortable with the statement
of SVD or felt that I understood it in a more than formal way. (I might add that many theoretical
linear algebra texts do not mention the SVD and many of my more theoretically minded colleagues do
not even recognize the term.) But a few days ago, a social scientist friend of mine asked me about
a problem he was interested in; one that involved the SVD in an essential way. After thinking about
it for a while, I realized that SVD can be reformulated as a statement about linear transformations
that, to me at least, seems a lot more conceptual and geometric:
If $T$ is a linear map, say of rank $r$, between finite dimensional inner-product spaces $V$ and
$W$, then there are orthonormal bases $v_1, \ldots, v_m$ for $V$ and $w_1, \ldots, w_n$ for $W$
and $r$ positive numbers $s_1 \ge s_2 \ge \ldots \ge s_r$, such that $T v_i$ equals $s_i w_i$ if
$i \le r$ and equals zero if $i > r$.
I certainly realize that this is a pretty obvious reformulation of SVD, once you see it (and
those poor misguided souls who prefer matrices to linear transformations may even see it as
a step backwards :-), but my question is whether there is some standard source for this
reformulation that I can reference.
Best Answer
I just looked in Wikipedia (http://en.wikipedia.org/wiki/Singular_value_decomposition). There is a very thorough discussion, including a section "Geometric meaning" in which your interpretation is clearly explained. Well, there $K^n$, where $K = \mathbb R$ or $K = \mathbb C$ is used, instead of arbitrary inner product spaces, but I suppose you'll agree that the difference is not essential.