I have two 3x3 rotation matrices $R_1$ and $R_2$. Let's say $R_1$ is computed by some (stochastic gradient descent) optimization algorithm and $R_2$ is the target. I am looking for a measurement of the error between $R_1$ and $R_2$.
Naturally, I could compute the rotation angle $\theta$ it takes to rotate $R_1$ into $R_2$ and minimize this angle. Similar questions have been asked here:
However, my problem is that $R_1$ might not be a valid rotation matrix, especially in the beginning of the optimization. I see two ways to correct for that:
- Compute $\theta$ by converting the rotation matrices into quaternions and then compute the error as shown e.g. here (Quaternion distance). When converting $R_1$ to a quaternion, divide by its norm to ensure it is unit and thus a valid rotation. Problem: the formulas to convert a matrix to a quaternion assume that the matrix is a valid rotation. Making the resulting quaternion unit guarantees that the quaternion is a valid rotation, but I don't know how ''far away'' the resulting rotation is and this could be a problem for the optimization.
- First extract the closest rotation matrix $R_1'$ from $R_1$ by using the SVD as explained here. Using this, I know that $R_1'$ is the closest valid rotation matrix measured by the Frobenius norm, but if possible I would like to avoid having an SVD in the computation of my loss.
The last possibility is to just not care about the validity of $R_1$ and hope that the SGD is still able to converge. This also makes sense to me, because when $R_1$ is not a valid rotation, defining an angle-dependent metric on that matrix is an ill-posed problem and thus it is not clear how to handle it. I will try this for sure, but I'm also interested in more technical details about the general problem I described and/or what would be the recommended way for this scenario.
Best Answer
Regarding efficiency of computation, polar decomposition of a matrix can be used to extract the rotation matrix $R$ from $R_1$, then for compare the two rotations you can compute the error as $E = \|R R_2^{-1} - I\|_F^2$ is a valid measure.
Using quaternions is a very similar approach, extracting the quaternion $Q_1$ from $R_1$ gives similar effect as taking polar decomposition, then computing $E = \|Q_1 Q_2^{-1} - 1\|^2$ is also valid measure.
If $R_1$ is not very far from rotation, you can compute directly $E = \|R_1 R_2^{-1} - I\|_F^2$, skipping the polar decomposition.