MATLAB: Large error in sparse 3D point cloud

Question

I am working on sparse reconstruction using a calibrated stereo pair. This is the approach I have taken step by step (thanks to Dima):

1- I Calibrated my stereo cameras using the Stereo Camera Calibrator app.

2- I Took a pair of stereo images and Undistorted each image.

3- I Detect, extract, and match point features.

4- I Use the triangulate function to get 3D coordinates of the matched points by passing the stereoParametes object into triangulate. The resulting 3D coordinates are with respect to the optical center of camera 1 and it is in millimeters.

The problem is that the point clouds seems to be warped and curved towards the edges of the image. at first it seemed like a barrel distortion of the lenses to me. so I recalibrated the bumblebee XB3 cameras using MATLAB camera calibrator app. but this time I used 3 radial distortion coefficients and also included tangential and skew parameters. but the results are the same. I also tried Caltech's camera calibration toolbox but it had the same results as MATLAB. also another problem is that the Z values in the point cloud are all negative but I am thinking that might be coming from the fact that I am using right camera as camera 1 and left camera as camera 2 as oppose to what MATLAB's coordinate system is in the link attached.

I have attached couple of pictures of 3D point cloud from both sparse and dense 3D reconstruction. I am not interetsed in Dense 3D but just wanted to do it to see if the problem still exist which it does. I believe that means the main problem is with the images and camera calibration rather than algorithms.

Now my questions are:

1- What is the main reason/reasons for having warped/curved 3D point clouds? is it camera calibration only or other steps may introduce error as well? how can I check on that?

2- Can you suggest another camera calibration toolbox besides MATLAB's and Caltech's? maybe one that is more suitable for radial distortions?

Thanks

simillar question

coordinate system

Answer 1

Hi Luca,

Thanks for your kind words.

You are correct about the negative values. If camera 1 is on the right, then that is what you would get.

As for the curving of the point cloud, that is harder to diagnose. I have some questions for you. How many image pairs did you use for calibration? What kind of reprojection errors did you get? What is the resolution of your camera? When you undistorted the images, did they look reasonable to your eyes? Or was there still some distortion present?

What I am getting at is that you may need to re-calibrate using more images with the checkerboard close to the edges of the image, to get a better estimate of the distortion coefficients.

Another question: what 'OutputView' did you use for the undistortImage? And if you used 'full' or 'valid', did you remember to adjust your points for the shift of the origin?

By the way, next time you have a question on this topic, please specify "Computer Vision System Toolbox" under "Products". This way you will probably get an answer more quickly.