I'm starting to look at some lectures on the SVD. The lectures start out by saying that in $$\mathbf{y}=A\mathbf{x}$$ the transformation representing $A$ is only doing rotation and stretching.
- The matrix for rotation is $$ \begin{bmatrix}\cos\theta&-\sin\theta\\\sin\theta&\cos\theta\end{bmatrix}$$
- The matrix for stretching is $$ \begin{bmatrix}\alpha&0\\0&\alpha\end{bmatrix}$$
In the past I've learnt that transformations can be rotations, reflections, scaling, shears, and flattenings. However the idea that any transformation can be decomposed into a rotation and a stretch is new to me. Is there a clear and/or intuitive way of explaining this?
Best Answer
One major theme in the study of matrix algebra is that of matrix decompositions. The idea is to take a matrix $A$ (either arbitrary or satisfying some reasonable conditions) and decompose it as the product of two or three matrices of a simpler form. For example, the so-called QR decomposition writes an arbitrary matrix $A$ as
$$A = QR$$
Where $Q$ is orthogonal (a composition of rotations + reflections) and $R$ is upper-triangular. In two-dimensions, $Q$ is either rotation or reflection.
In the subject of SVD, an important matrix decomposition is the aptly-named singular value decomposition. This writes an arbitrary matrix $A$ in the form
$$A = UDV^{H}$$
where $D$ is a nonnegative, diagonal matrix whose entries are called the singular values, $U$ and $V$ are orthogonal (unitary in the complex case), and $V^{H}$ is the transpose of $V$ (or conjugate-transpose if complex entries are allowed). A diagonal matrix can be thought of as stretching a basis by the corresponding singular values, and an orthogonal matrix is a rotation or reflection. Thus, this decomposition sees $A$ as a composition of a sort of rotation (transposed), a stretching, and finally another rotation.
You can read more about the singular value decomposition here.