[Math] Decompose distortion affected homography matrix

geometrynoisetransformation

I am working on a system that finds homography between images taken by moving (shaking) camera with rolling shutter and map. The map is orthogonal image of flat 2D plane and the camera images are taken from arbitrary angles and distances. Homography is found as geometrical transformation between two planes using OpenCV's function findHomography.

I tried to decompose calculated homography matrix (which 3×3 matrix) to obtain camera/object pose (using known camera intrinsic matrix) represented by 3×4 matrix [R|t] where R is rotation matrix and t is translation vector, using simple algorithm described here. After some experiments I found that the resulting pose estimations are inacurate. So I suppose that calculated homography matrix is not only composed by rotation, translation and camera intrinsic matrix, but it also contains transformation caused by rolling shutter distortion – shear and dilation in y-axis.

So, my idea is, that the homography H is actually composed as follows:

H = D * M * [R|t]

where D is distortion matrix which consist purely by shear and dilation in y-axis and M is camera intrinsic matrix which has following format:
$$
\begin{bmatrix}
f_x & 0 & c_x\\
0 & f_y & c_y\\
0 & 0 & 1\\
\end{bmatrix}
$$

Is there an algorithm to decompose homography matrix into all three matrices using just known homography matrix and camera intrinsic matrix?

Best Answer

Homography cannot decomposed to the from you wrote. For example the translation $t$ and $c_x,c_y$ cannot be determined since only they sum is known.

See page 8: in this paper