Methods, devices and computer program products for global bundle adjustment of 3D images

1. A method for estimation of a Three-Dimensional, 3D, map and a plurality of poses from a plurality of Two-Dimensional, 2D, images, the method comprising:

performing an initialization of a plurality of 3D points in the 3D map based on the plurality of 2D images;

reprojecting ones of the 3D points onto ones of the 2D images to obtain reprojection points;

determining respective reprojection errors of ones of the reprojection points;

determining a change in steepness of a curve corresponding to the reprojection errors and/or a median and/or mean reprojection error of the reprojection errors of the reprojection points;

determining a robust statistic parameter associated with a weighting function based on the respective reprojection errors, wherein the robust statistic parameter is determined based on the change in steepness of the curve corresponding to the reprojection errors and/or a scaling of the median and/or mean reprojection error; and

performing bundle adjustment to update the 3D map and/or update the plurality of poses, based on the robust statistic parameter and the weighting function,

wherein the performing bundle adjustment comprises iteratively updating the 3D map and the plurality of poses based on updated reprojection errors and an updated robust statistic parameter associated with an updated weighting function.

2. The method of claim 1 , wherein the updated robust statistic parameter for a current iteration of the bundle adjustment is different from the robust statistic parameter for a previous iteration of the bundle adjustment.

3. The method of claim 1 , wherein the updated robust statistic parameter is reduced with subsequent iterations of the bundle adjustment.

4. The method of claim 1 , wherein the reprojecting ones of the 3D points comprises a first reprojection, wherein the bundle adjustment comprises a first bundle adjustment, and wherein the iteratively updating the 3D map and the plurality of poses comprises:

performing a second reprojection by reprojecting the ones of updated 3D points onto ones of the 2D images to obtain updated reprojection points;

determining respective updated reprojection errors of ones of the updated reprojection points;

determining the updated robust statistic parameter associated with the updated weighting function based on the respective updated reprojection errors; and

performing a second bundle adjustment to update the 3D map and update the plurality of poses, based on the updated robust statistic parameter and the updated weighting function.

5. The method of claim 1 , further comprising:

reducing 3D drift in the 3D map and/or the plurality of poses based on the iteratively updating the 3D map and the plurality of poses.

6. The method of claim 1 , wherein the performing the initialization of the plurality of 3D points in the 3D map comprises:

determining respective poses of the plurality of poses for ones of the plurality of 2D images.

7. The method of claim 1 , wherein the performing the initialization of the plurality of 3D points in the 3D map comprises:

selectively determining that a 2D image of the plurality of 2D images is a key frame of a plurality of key frames.

8. The method of claim 7 ,

wherein the selectively determining that the 2D image of the plurality of 2D images is the key frame comprises determining that the 2D image of the plurality of 2D images is the key frame, responsive to a change in a pose from another 2D image of the plurality of 2D images being greater than an angle threshold, and

wherein the performing bundle adjustment comprises performing bundle adjustment on the plurality of key frames of the plurality of 2D images to update the 3D map and update the plurality of poses.

9. The method of claim 1 , further comprising:

estimating at least one of focal length, skew, principal points, or distortion from the plurality of the 2D images.

10. The method of claim 1 , wherein the respective reprojection errors comprise respective distances of respective reprojection points to corresponding 2D points in ones of the 2D images.

11. The method of claim 1 ,

wherein the weighting function selectively provides a greater weight to a second one of the reprojection points than to a first one of the reprojection points, responsive to a first reprojection error associated with the first one of the reprojection points being greater than a second reprojection error associated with the second one of the reprojection points.

12. The method of claim 1 , further comprising:

minimizing a sum of the reprojection errors of the reprojection points based on the performing bundle adjustment.

13. The method of claim 1 ,

wherein the weighting function comprises a quadratic function for first ones of the reprojection errors that are less than a threshold reprojection error, and

wherein the weighting function comprises a linear function for second ones of the reprojection errors that are greater than the threshold reprojection error.

14. The method of claim 1 , further comprising:

receiving the 2D images from a camera.

15. A computer program product for operating an image capturing system, the computer program product comprising a non-transitory computer readable storage medium having computer readable program code embodied in the medium that when executed by a processor causes the processor to perform the method of claim 1 .

16. An imaging system for processing images, the imaging system comprising:

a processor; and

a memory coupled to the processor and storing computer readable program code that when executed by the processor causes the processor to perform operations comprising:

performing an initialization of a plurality of 3D points in a 3D map based on a plurality of 2D images;

reprojecting ones of the 3D points onto ones of the 2D images to obtain reprojection points;

determining respective reprojection errors of ones of the reprojection points;

determining a change in steepness of a curve corresponding to the reprojection errors and/or a median and/or mean reprojection error of the reprojection errors of the reprojection points;

determining a robust statistic parameter associated with a weighting function based on the respective reprojection errors, wherein the robust statistic parameter is determined based on the change in steepness of the curve corresponding to the reprojection errors and/or a scaling of the median and/or mean reprojection error; and

performing bundle adjustment to update the 3D map and update the plurality of poses, based on the robust statistic parameter and the weighting function,

wherein the performing bundle adjustment comprises iteratively updating the 3D map and the plurality of poses based on updated reprojection errors and an updated robust statistic parameter associated with an updated weighting function.

17. The image processing system of claim 16 , wherein the updated robust statistic parameter for a current iteration of the bundle adjustment is different from the robust statistic parameter for a previous iteration of the bundle adjustment.