I have $2$ images rotated by $60^\circ$ to each other with different center of rotation. The here presented matrices are affine transformation matrices derived from OpenCV: https://docs.opencv.org/4.x/d4/d61/tutorial_warp_affine.html
If I prerotate the second image about $60^\circ$ so just a translational offset remains I get the following matrix:
$$
\begin{bmatrix}
9.99509\cdot10^{-1} & 2.36089\cdot10^{-3} & 71.3039 \\
-2.36089\cdot10^{-3} & 9.99509\cdot10^{-1} & 52.4813
\end{bmatrix}
$$
The translational offset of $71.3$ pixels and $52.4$ pixels is correct. Without prerotation I get the following matrix:
$$
\begin{bmatrix}
4.9799\cdot10^{-1} & 8.66867\cdot10^{-3} & -114.5329 \\
-8.66867\cdot10^{-3} & 4.9799\cdot10^{-1} & 752.35292
\end{bmatrix}
$$
From the second matrix I get the correct angle, but is there a way to get also the rotational corrected translational offset?
I tried this approach but I assume it only works if the the images share the same rotation center:
$$
\left\{
\begin{aligned}
x' &= ax + cy + e \\
y' &= bx + dy + f
\end{aligned}
\right.
$$
The core of my question is: What is the relation between both matrices? The first matrix is already corrected by rotation (angle is around 0) so I can directly extract the translation.
In the second matrix the rotation and translation is connected to each other. I can calculate the angle out of it, but how do I apply that angle on the matrix so that I get the pure translation such as in the first matrix? So how do I get the first matrix out of my second matrix?
My interpretation of the affine transformation matrix is, that it combines rotation and translation, but rotation needs to be applied first. So independet on which rotation point I apply the rotation on the image, there is always a point (0,0) which doesnt get rotated and for which the translation parameters of the matrix should directly can be used.
Many thanks in advance!
EDIT: Here is a minimal example. I found the definition of the rotation matrix: Equation. So I now want to try to build a rotation matrix with the angle I get from the image registration and then extract backwards via matrix multiplication the translation matrix. I think it should work but some small thing is missing. Strangly, I get good results by just calculating (via the resulting angle) the last column in the definition (see equation screenshot) and subtract that from the matrix I get via image registration. Since it doesnt work on my minimal example I had doubts since it only represents the rotation. So this way here if it would work should be more correct.
import numpy as np
import cv2 as cv, cv2
import matplotlib.pyplot as plt
import math
img=cv2.imread(r"\lena_std.tif",cv2.IMREAD_GRAYSCALE)
tx=0 #parameters to play which should be totally compensated by the code
ty=0
angle=60
rows,cols= img.shape
rot_mat = cv2.getRotationMatrix2D((cols/2,rows/2),angle,1)
rot_mat=np.vstack([rot_mat, np.array([0,0,1])])
trans_mat = np.array([
[1, 0, tx],
[0, 1, ty],
[0, 0, 1]
], dtype=np.float32)
M=trans_mat @ rot_mat
reference_image = img
test_image = cv2.warpAffine(img,M[:2,:3],(cols,rows))
def similarity_sift_flann_affine2D (reference_image, test_image):
img1=test_image.astype(np.uint8)
img2=reference_image.astype(np.uint8)
# Initiate SIFT detector
sift = cv.SIFT_create()
# find the keypoints and descriptors with SIFT
kp1, des1 = sift.detectAndCompute(img1,None)
kp2, des2 = sift.detectAndCompute(img2,None)
FLANN_INDEX_KDTREE = 1
index_params = dict(algorithm = FLANN_INDEX_KDTREE, trees = 5)
search_params = dict(checks = 50)
flann = cv.FlannBasedMatcher(index_params, search_params)
matches = flann.knnMatch(des1,des2,k=2)
# store all the good matches as per Lowe's ratio test.
good = []
for m,n in matches:
if m.distance < 0.7*n.distance: #Lowes ratio test
good.append(m)
MIN_MATCH_COUNT=10
if len(good)>=MIN_MATCH_COUNT:
src_pts = np.float32([ kp1[m.queryIdx].pt for m in good ]).reshape(-1,1,2)
dst_pts = np.float32([ kp2[m.trainIdx].pt for m in good ]).reshape(-1,1,2)
M, mask = cv2.estimateAffinePartial2D(src_pts, dst_pts)
#M,mask = cv.estimateAffine2D(src_pts, dst_pts)
else:
print( "Not enough matches are found - {}/{}".format(len(good), MIN_MATCH_COUNT) )
matchesMask = None
return M
M1=similarity_sift_flann_affine2D (reference_image, test_image)
sizeImg1=img.shape
u, _, vh = np.linalg.svd(M[0:2, 0:2])
R = u @ vh
angle2 = math.atan2(R[1,0], R[0,0])
angle2 =np.rad2deg(angle2)
cor_rot_mat = cv2.getRotationMatrix2D((cols/2,rows/2),-angle2,1)
cor_rot_mat=np.vstack([cor_rot_mat, np.array([0,0,1])])
M1=np.vstack([M1, np.array([0,0,1])])
#cor_trans_mat= np.linalg.inv(cor_rot_mat) @ M1
cor_trans_mat= M1 @ np.linalg.inv(cor_rot_mat)
print("angle diff", str(np.abs(angle2)-np.abs(angle)))
print("tx diff", str(np.abs(tx)-np.abs(cor_trans_mat[0][2])))
print("ty diff", str(np.abs(ty)-np.abs(cor_trans_mat[1][2])))
Best Answer
Your object is an image. In an image the $x$ axis extends horizontally from the top left corner to the right of the image, and the $y$ axis extends vertically from the top left corner downward to the bottom of the image. As the equation you provided states, the result of rotating a pixel $p=(x, y)$ in the original image about the center $C = (C_x, C_y)$ by angle $\theta$ counter clockwise is the pixel $p' =(x',y')$ where
$ p' = C + R (p - C) = R p + (I - R) C $
where
$ R = \begin{bmatrix} \cos \theta && \sin \theta \\ - \sin \theta && \cos \theta \end{bmatrix} $
Therefore, if we use $3 \times 1$ vectors for $p$ and $p'$ as follows
$ P = \begin{bmatrix} p \\ 1 \end{bmatrix} , \hspace{30pt} P' = \begin{bmatrix} p' \\ 1 \end{bmatrix} $
Then the relation between $P'$ and $P$ is
$ P' = T P $
where $T $ is the linear transformation matrix and is given by
$ T = \begin{bmatrix} R && (I - R) C \\ 0^T && 1 \end{bmatrix} $
Writing out $T$ fully, we get
$ T = \begin{bmatrix} \cos \theta && \sin \theta && (1 - \cos \theta) \ C_x - \sin \theta \ C_y \\ - \sin \theta && \cos \theta && \sin \theta \ C_x + (1 - \cos \theta ) \ C_y \\ 0 && 0 && 1 \end{bmatrix} $
Now if we shift the image to the right by $S_x$ and downward by $S_y$, then this corresponds to the shift transformation matrix $S$ given by
$ S = \begin{bmatrix} 1 && 0 && S_x \\ 0 && 1 && S_y \\ 0 && 0 && 1 \end{bmatrix} $
And now the overall transformation is
$ T_1 = S T $
And this equals
$ T_1 = \begin{bmatrix} \cos \theta && \sin \theta && (1 - \cos \theta) \ C_x - \sin \theta \ C_y + S_x \\ - \sin \theta && \cos \theta && \sin \theta \ C_x + (1 - \cos \theta ) \ C_y + S_y \\ 0 && 0 && 1 \end{bmatrix} $
If, however, you shift first then rotate, then we have to flip the order of multiplication of $S$ and $T$, so that in this case we'll have the overall transformation matrix as follows
$T_2 = T S = \begin{bmatrix} \cos \theta && \sin \theta && \cos \theta \ S_x + \sin \theta \ S_y + (1 - \cos \theta) \ C_x - \sin \theta \ C_y \\ - \sin \theta && \cos \theta && - S_x \ \sin \theta + S_y \ \cos \theta + \sin \theta \ C_x + (1 - \cos \theta) \ C_y \\ 0 && 0 && 1 \end{bmatrix} $
Now to answer your question, I will assume that you know the center of rotation $(C_x, C_y)$, and you want to find the translation $(S_x, S_y)$. If you rotated first, then shifted, then you have to use $T_1$ to conclude that
$ S_x = A - (1 - \cos \theta ) \ C_x + \sin \theta \ C_y $
$ S_y = B - \sin \theta \ C_x - (1 - \cos \theta) \ C_y $
where the last column of you transformation matrix is
$ \begin{bmatrix} A \\ B \\ 1 \end{bmatrix} $
On the other hand, if you shifted first, then rotated, you have to use $T_2$ to write
$ A = \cos \theta \ S_x + \sin \theta \ S_y + (1 - \cos \theta) \ C_x - \sin \theta \ C_y $
$ B = - S_x \ \sin \theta + S_y \ \cos \theta + \sin \theta \ C_x + (1 - \cos \theta) \ C_y $
These are two linear equations in $S_x$ and $S_y$, and readily solvable to
$ \begin{bmatrix} S_x \\ S_y \end{bmatrix} = \begin{bmatrix} \cos \theta && - \sin \theta \\ \sin \theta && \cos \theta \end{bmatrix} \begin{bmatrix} A - (1 - \cos \theta ) \ C_x + \sin \theta \ C_y \\ B - \sin \theta \ C_x - (1 - \cos \theta ) \ C_y \end{bmatrix} $
I hope this answers your question.