[Math] Why the SVD is named so…

Question

The SVD stands for Singular Value Decomposition. After decomposing a data matrix $\mathbf X$ using SVD, it results in three matrices, two matrices with the singular vectors $\mathbf U$ and $\mathbf V$, and one singular value matrix whose diagonal elements are the singular values. But I want to know why those values are named as singular values. Is there any connection between a singular matrix and these singular values?

Answer 1

From On the Early History of the Singular Value Decomposition by Pete Stewart:

The term "singular value" seems to have come from the literature on integral equations. A little after the appearance of Schmidt's paper, Bateman refers to numbers that are essentially the reciprocals of the eigenvalues of the kernel as singular values. Picard combined Schmidt's results with Riesz's theorem on the strong convergence of generalized Fourier series to establish a necessary and sufficient condition for the existence of solutions of integral equations.

In a later paper on the same subject, he notes that for symmetric kernels Schmidt's eigenvalues are real and in this case (but not in general) he calls them singular values. By 1937, Smithies was referring to singular values of an integral equation in our modern sense of the word. Even at this point, usage had not stabilized. In 1949, Weyl speaks of the "two kinds of eigenvalues of a linear transformation," and in a 1969 translation of a 1965 Russian treatise on nonselfadjoint operators Gohberg and Krein refer to the "s-numbers" of an operator.

See the paper for more fascinating accounts on how SVD came to be, even before the seminal paper of Golub/Kahan.