MATLAB: Image stitching: ideas to solve singular or badly scaled matrix

Computer Vision Toolboximageimage stitchingMATLABmatrixpanoramasingularity

I'm trying to do image mosaicing according to this example of MATLAB:
Feature Based Panoramic Image Stitching
My images are captured in an approximate vertical column. What the code does, is to find all the transformation with respect to the first image. i.e. :
tforms(n).T = tforms(n-1).T * tforms(n).T;
Then, it starts stitching (warping) the images based on the obtained transformations above. The code works well when stitching 3-4 images. But starts to give this popular warning after that! I'm trying to stitch about 25 images and with the start of this warning the images are not stitched properly:
Warning: Matrix is close to singular or badly scaled. Results may be inaccurate. RCOND =  2.679189e-08. 
> In affine2d>affine2d.get.Tinv at 339
  In affine2d>affine2d.transformPointsInverse at 214
  In imwarp>remapAndResampleGeneric2d at 248
  In imwarp>remapPointsAndResample at 188
  In imwarp at 173
Could anyone suggest how to solve this problem? From physical point of view, this procedure must work, but how to fix this mathematical shortcoming? To give you an idea of the resulting error, here you can see one ill-conditioned transformation matrix and the final mosaic:
1.0e+03 *
      0.0010   -0.0000         0
      0.0000    0.0010         0
     -0.0592   -5.8776    0.0010

Best Answer

Hi Mona,
the problem here is that each time the transformation is estimated, errors are introduced. These errors accumulate, which is why the stitching works for a few images, but not for 25.
One thing you can try to improve the mosaic, is to start from the middle image, rather than from the first one. Let's say you have 7 images. First build a mosaic using images 4, 5, 6, and 7. Then build a mosaic using images 4, 3, 2, and 1. Then stitch the two mosaics. This way, in both mosaics you are not moving two far from the starting image, so you should have a smaller error.
The other alternative is to use the Optimization Toolbox to do a global optimization over the transformations to minimize the geometric errors.
Related Question