This question follows-on from Correct the Fourier transform of a shear-distorted image? ($x_{new} = x + \alpha y$). I have a group of distorted $x, y$ points and have found a matrix $A$ that corrects their positions:
$$
\begin{bmatrix}
x' \\
y' \\
\end{bmatrix} =
\begin{bmatrix}
a_{xx} & a_{xy} \\
a_{yx} & a_{yy} \\
\end{bmatrix}
\begin{bmatrix}
x \\
y \\
\end{bmatrix}
$$
I now want to break this up into three separate terms, one for pure scaling or magnification $M$, one for pure rotation $R$, and one for shear $S$. I think they will look like this:
$$M = \begin{bmatrix}
m_x & 0\\
0 & m_y \\
\end{bmatrix}
$$
$$R = \begin{bmatrix}
\cos \theta & -\sin \theta \\
\sin \theta & \cos \theta \\
\end{bmatrix}
$$
$$S = \begin{bmatrix}
1 & s_{xy} \\
s_{yx} & 1 \\
\end{bmatrix}
$$
I see there are answers to
- Separating Out Parts of a Matrix (Translation, Rotation, Scaling)
- Is it true that any matrix can be decomposed into product of rotation,reflection,shear,scaling and projection matrices?
and I recognize that those answers may apply to my problem, but they are beyond me; I need a simpler, more applied answer, or at least assistance getting there.
Q1: I can imagine setting $A$ equal to the product, but I'm worried if the order matters. Will this order work? Would any order work equally as well?
$$A = R \ S \ M $$
Q2: I now have five parameters instead of four. Since I've separated out rotation, should I now use only one shear, i.e. I have to choose only one of $s_xy$ or $s_yx$ and set the other to zero? Or just use $s_{xy} = s_{yx} = s$?
$$S = \begin{bmatrix}
1 & s \\
s & 1 \\
\end{bmatrix}
$$
Best Answer
You have already observed the bad effect if you use $-s$ and $s$ in the off-diagonal terms of your "shear" matrix (making an antisymmetric matrix) -- the matrix doesn't just look like a rotation matrix, it actually is a combined rotation and scaling.
Changing the form of the matrix to a symmetric matrix does not help, because $$ \begin{bmatrix} 1 & s \\ s & 1 \end{bmatrix} = \begin{bmatrix} \cos\left(-\frac\pi4\right) & -\sin\left(-\frac\pi4\right) \\ \sin\left(-\frac\pi4\right) & \cos\left(-\frac\pi4\right) \end{bmatrix} \begin{bmatrix} 1 - s & 0 \\ 0 & 1 + s \end{bmatrix} \begin{bmatrix} \cos\left(\frac\pi4\right) & -\sin\left(\frac\pi4\right) \\ \sin\left(\frac\pi4\right) & \cos\left(\frac\pi4\right) \end{bmatrix}. $$ That is, the effect of the symmetric matrix is what we get by rotating $\frac\pi4$ radians counterclockwise, dilating by a factor $1-s$ in the direction of the $x$ axis and $1+s$ in the direction of the $y$ axis, and rotating back by $\frac\pi4$ radians clockwise. Equivalently, it is a dilation by $1-s$ in the direction of the vector $[1,-1]^T$ and a dilation by $1+s$ in the direction of the vector $[1,1]^T.$
As a more concrete way of looking at it, just observe what the transformation does to the unit square: it distorts it into a rhombus for any value of $s.$ This is just stretching, compressing, or even reflecting it along its two diagonals; there is no shear at all.
On the other hand, if you look at the effect on the unit square of a transformation by the matrix $$ \begin{bmatrix} 1 & s \\ 0 & 1 \end{bmatrix}, $$ it is clearly a shear transformation: the vertices $(0,0)$ and $(1,0)$ are unaffected, while the vertices $(0,1)$ and $(1,1)$ are both moved $s$ units to the right.
Now let's see how we can decompose an arbitrary $2\times2$ matrix into matrices for rotation, scaling, and shear. We start with $$ A = \begin{bmatrix} a & b \\ c & d \end{bmatrix}. $$ (Since there are only four entries, I find it simple enough to keep track of them by name rather than by subscripts, and the formulas will be less "busy" this way.) Note what happens if we apply a rotation of $\theta$ radians to this matrix: $$ \begin{bmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{bmatrix} \begin{bmatrix} a & b \\ c & d \end{bmatrix} = \begin{bmatrix} a \cos\theta - c \sin\theta & b \cos\theta - d \sin\theta \\ c \cos\theta + a \sin\theta & d \cos\theta + b \sin\theta \end{bmatrix}. $$ Let $\theta = -\operatorname{atan2}(c, a)$ where $\operatorname{atan2}$ is the two-parameter arc tangent function (see this answer for a definition). Then $\sin\theta = -\frac{c}{\sqrt{a^2+c^2}}$ and $\cos\theta = \frac{a}{\sqrt{a^2+c^2}}$, so $c \cos\theta + a \sin\theta = 0.$ Define a rotation matrix $Q$ (not $R$; let's reserve the name $R$ for later) by $$ Q = \begin{bmatrix}\cos\theta & -\sin\theta \\ \sin\theta & \cos\theta\end{bmatrix}. $$ Then $$ Q A = \begin{bmatrix} a \cos\theta - c \sin\theta & b \cos\theta - d \sin\theta \\ 0 & d \cos\theta + b \sin\theta \end{bmatrix}. $$ Suppose $a \cos\theta - c \sin\theta \neq 0$ and $d \cos\theta + b \sin\theta \neq 0.$ (The alternative is that the matrix $QA$ is a projection and therefore so is the original matrix $A$.) Define a scaling matrix $N$ (saving the name $M$ for later) by $$ N = \begin{bmatrix} \frac{1}{a \cos\theta - c \sin\theta} & 0 \\ 0 & \frac{1}{d \cos\theta + b \sin\theta} \end{bmatrix}. $$ Then $$ NQA = \begin{bmatrix} 1 & \frac{b\cos\theta - d\sin\theta}{a\cos\theta - c\sin\theta}\\ 0 & 1 \end{bmatrix}. $$ This is a shear. So let $NQA = S$ and multiply both sides of the equation by $Q^{-1}N^{-1}$; then $$ A = Q^{-1} N^{-1} S = RMS $$ where \begin{align} R &= Q^{-1} = \begin{bmatrix} \cos\theta & \sin\theta \\ -\sin\theta & \cos\theta \end{bmatrix} = \begin{bmatrix} \frac{a}{\sqrt{a^2 + c^2}} & \frac{-c}{\sqrt{a^2 + c^2}} \\ \frac{c}{\sqrt{a^2 + c^2}} & \frac{a}{\sqrt{a^2 + c^2}} \end{bmatrix}, \\ M &= N^{-1} = \begin{bmatrix} a \cos\theta - c \sin\theta & 0 \\ 0 & d \cos\theta + b \sin\theta \end{bmatrix} = \begin{bmatrix} \sqrt{a^2 + c^2} & 0 \\ 0 & \frac{ad-bc}{\sqrt{a^2 + c^2}} \end{bmatrix}, \\ S &= \begin{bmatrix} 1 & \frac{b\cos\theta - d\sin\theta}{a\cos\theta - c\sin\theta}\\ 0 & 1 \end{bmatrix} = \begin{bmatrix} 1 & \frac{ab + cd}{a^2 + c^2} \\ 0 & 1 \end{bmatrix}. \end{align}
This is good for every matrix $A$ except the case where $a = c = 0,$ which is a projection and therefore presumably not what you would use to "correct" the positions of points. For completeness, however, if $a = c = 0$ and $b^2 + d^2 \neq 0$ we can write $$ A = \begin{bmatrix} 0 & b \\ 0 & d \end{bmatrix} = \begin{bmatrix} \frac{d}{\sqrt{b^2 + d^2}} & \frac{b}{\sqrt{b^2 + d^2}} \\ \frac{-b}{\sqrt{b^2 + d^2}} & \frac{d}{\sqrt{b^2 + d^2}} \end{bmatrix} \begin{bmatrix} 0 & 0 \\ 0 & \sqrt{b^2 + d^2} \end{bmatrix} \begin{bmatrix} 1 & s \\ 0 & 1 \end{bmatrix} $$ where $s$ is whatever you like, that is, $A=RMS$ where $R$ is a rotation, $M$ is a scaling matrix (where $m_x = 0$) and $S$ is a shear.
And in the final case, $a = b = c = d,$ obviously you can just set $M$ to the zero matrix (that is, $m_x = m_y = 0$) and use any shear and rotation you like.
So it is indeed always possible to decompose $A$ into a product of the form $RMS,$ even in the cases you probably don't care about.
This suggests that it is possible to use another sequence such as $RSM,$ and indeed that particular sequence is possible. In the $RMS$ derivation, the next step after zeroing out the lower left corner (via rotation) was to scale so that the diagonals are $1$ (which immediately gives you a shear matrix); for $RSM,$ let's use a shear to produce a diagonal matrix. That is, constructing $Q$ as before we have $$ Q A = \begin{bmatrix} a \cos\theta - c \sin\theta & b \cos\theta - d \sin\theta \\ 0 & d \cos\theta + b \sin\theta \end{bmatrix}. $$ A shear parallel to the $x$ axis will have no affect on the first column of this matrix, but the effect on the second column can be seen in $$ \begin{bmatrix} 1 & t \\ 0 & 1 \end{bmatrix} \begin{bmatrix} b \cos\theta - d \sin\theta \\ d \cos\theta + b \sin\theta \end{bmatrix} = \begin{bmatrix} t(d\cos\theta + b\sin\theta) + b\cos\theta - d\sin\theta \\ d \cos\theta + b \sin\theta \end{bmatrix}. $$ We need $t(d\cos\theta + b\sin\theta) + b\cos\theta - d\sin\theta = 0,$ which we get by setting $t = \frac{d\sin\theta - b\cos\theta}{d\cos\theta + b\sin\theta}.$
but the derivation might be a little more difficult to find (I have not tried) and we should expect the resulting formulas to be quite different. Then $$ TQA = \begin{bmatrix} a \cos\theta - c \sin\theta & 0 \\ 0 & d \cos\theta + b \sin\theta \end{bmatrix} $$ where $T$ is the shear matrix $$ T = \begin{bmatrix} 1 & \frac{d\sin\theta - b\cos\theta}{d\cos\theta + b\sin\theta}\\ 0 & 1 \end{bmatrix} = \begin{bmatrix} 1 & -\frac{ab + cd}{ad - bc} \\ 0 & 1 \end{bmatrix} $$ We see that $M = TQA$ is a scaling matrix, so we have $A = RSM$ where \begin{align} R &= Q^{-1} = \begin{bmatrix} \cos\theta & \sin\theta \\ -\sin\theta & \cos\theta \end{bmatrix} = \begin{bmatrix} \frac{a}{\sqrt{a^2 + c^2}} & \frac{-c}{\sqrt{a^2 + c^2}} \\ \frac{c}{\sqrt{a^2 + c^2}} & \frac{a}{\sqrt{a^2 + c^2}} \end{bmatrix}, \\ S &= T^{-1} = \begin{bmatrix} 1 & \frac{ab + cd}{ad - bc} \\ 0 & 1 \end{bmatrix}, \\ M &= \begin{bmatrix} a \cos\theta - c \sin\theta & 0 \\ 0 & d \cos\theta + b \sin\theta \end{bmatrix} = \begin{bmatrix} \sqrt{a^2 + c^2} & 0 \\ 0 & \frac{ad - bc}{\sqrt{a^2 + c^2}} \end{bmatrix}. \end{align} That is, the $RSM$ decomposition is just like the $RMS$ decomposition except for the shear factor.