[Math] Intuitive explanation of the singular values

singular valuessvd

I am really sorry if this a very naive question. There are some posts about Singular value decomposition (SVD) that explains SVD from the linear transformation perspective. I sort of understand that. However, I don't understand intuitively what singular values do. According to How can you explain the Singular Value Decomposition to Non-specialists? , a large singular value will indicate that the contribution of the corresponding transformation. I don't really understand what does that contribution means?

According to Understanding singular value decomposition, one geometric interpretation of the singular values of a matrix is the following. Suppose A is a $m×n$ matrix (real-valued, for simplicity). Think of it as a linear transformation $ℝ^n→ℝ^m$ in the usual way. Now take the unit sphere S for $ℝ^n$. Being a linear transformation, A maps S to an ellipsoid in $ℝ^m$. The lengths of the semi-axes of this ellipsoid are precisely the non-zero singular values of A. The zero singular values tell us what the dimension of the ellipsoid is going to be: $n$ minus the number of zero singular values.

If I understand correctly, according to above, the zero singular values are used to determine the dimension of the transformed space. How can the positive singular values be interpreted?

Best Answer

Here are answers to your two questions:

  1. A large singular value means, using the geometric interpretation, that the axis of the ellipsoid will be significantly longer than the corresponding axis of the sphere. More generally, the portion of the linear transformation of a vector from $\mathbb R^n$ to $\mathbb R^m$ corresponding to a large singular value is significant.

  2. In addition to what you describe in your question, positive singular values can be used to determine the effective rank of a matrix A by counting small values as zeros. That is often done when computing the SVD numerically.

    If A does have small singular values, it is advantageous in some applications to replace A = UDV * with UD'V * where D' is the same as D except that small singular values are changed to zero.

    See the Wikipedia items "Range, null space and rank" and "Low-rank matrix approximation."

Related Question