Method for estimating a camera motion and for determining a three-dimensional model of a real environment

1. A method for estimating a camera motion and for determining a three-dimensional model of an environment, comprising the steps of:

obtaining a set of reference two-dimensional imaged points captured by a camera at a first camera pose and reference depth samples associated to the set of reference two-dimensional imaged points;

determining a three-dimensional model of the environment using the reference depth samples and intrinsic parameters of the camera;

providing a set of current two-dimensional imaged points captured by the camera at a second camera pose and current depth samples associated to the set of current two-dimensional imaged points and determining a current three-dimensional model using the current depth samples and the intrinsic parameters;

estimating a camera motion between the first camera pose and the second camera pose by means of an image registration method;

determining based on the estimated camera motion a similarity measure between the three-dimensional model and the current three-dimensional model;

in response to determining that the similarity measure meets a first condition, updating the three-dimensional model of the environment using at least one point of the current three-dimensional model; and

in response to updating the three-dimensional model using at least one point of the current three-dimensional model, adding the set of current two-dimensional imaged points to the set of reference two-dimensional imaged points.

2. The method according to claim 1 , further comprising repeating providing a set of current two-dimensional points, estimating, and determining a similarity measure, using the three-dimensional model and the set of reference two-dimensional imaged points resulting from a previous iteration of providing a set of current two-dimensional points, estimating, and determining a similarity measure.

3. The method according to claim 2 , further comprising recording the similarity measure data over time and if it is determined that the similarity measure data in a defined time frame meets a second condition, the three-dimensional model is no longer updated and the current set of two-dimensional imaged points are no longer added to the reference set of two-dimensional imaged points.

4. The method according to claim 2 , further comprising determining a frequency of the similarity measure meeting the first condition and if it is determined that the frequency of updates falls below a defined threshold, the three-dimensional model is no longer updated and the current set of two-dimensional imaged points are no longer added to the reference set of two-dimensional imaged points.

5. The method according to claim 1 , wherein determining the similarity measure further comprises rendering the three-dimensional model into a coordinate system of the current three-dimensional model and determining the similarity measure between the three-dimensional model and the current three-dimensional model.

6. The method according to claim 1 , wherein the similarity measure is indicative of an overlap between the three-dimensional model and the current three-dimensional model.

7. The method according to claim 1 , further comprising determining whether an associated three-dimensional point exists in the three-dimensional model for at least one point in the current three-dimensional model.

8. The method according to claim 7 , further comprising determining whether a difference between the depth of the at least one point in the current three-dimensional model and the depth of the determined associated three-dimensional point in the three-dimensional model is above a defined threshold.

9. The method according to claim 7 , wherein the similarity measure is defined such that a number of points in the current three-dimensional model for which no associated three-dimensional point in the three-dimensional model is determined must exceed a certain threshold.

10. The method according to claim 7 , wherein only points in the current three-dimensional model for which no associated three-dimensional point in the three-dimensional model is determined are used for updating the three-dimensional model.

11. The method according to claim 1 , further comprising:

determining, based on the estimated camera motion, a depth of a three-dimensional point in the three-dimensional model and the depth of the three-dimensional point from the current depth samples associated to the set of current two-dimensional imaged points;

determining a refined depth of the three-dimensional point from the depth of the three-dimensional point in the three-dimensional model and the depth of the associated three-dimensional point in the current three-dimensional model; and

updating the depth of the three-dimensional point in at least one of the current three-dimensional point cloud, and in the three-dimensional model with the determined refined depth.

12. A system for estimating a camera motion and for determining a three-dimensional model of an environment, comprising:

a processor; and

a memory coupled to the processor and comprising computer code which, when executed by the processor, causes the system to:

obtain a set of reference two-dimensional imaged points captured by a camera at a first camera pose and reference depth samples associated to the set of reference two-dimensional imaged points;

determine a three-dimensional model of the environment using the reference depth samples and intrinsic parameters of the camera;

provide a set of current two-dimensional imaged points captured by the camera at a second camera pose and current depth samples associated to the set of current two-dimensional imaged points and determining a current three-dimensional model using the current depth samples and the intrinsic parameters;

estimate a camera motion between the first camera pose and the second camera pose by means of an image registration method;

determine based on the estimate of the camera motion a similarity measure between at least one three-dimensional point in the three-dimensional model and an associated three-dimensional point in the current three-dimensional model;

in response to determining that the similarity measure meets a first condition, updating the three-dimensional model of the environment using at least one point of the current three-dimensional model; and

in response to updating the three-dimensional model using at least one point of the current three-dimensional model, adding the set of current two-dimensional imaged points to the set of reference two-dimensional imaged points.

13. The system according to claim 12 , wherein the associated three-dimensional point in the current three-dimensional model is determined based on the estimated camera motion.

14. The system according to claim 12 , wherein the similarity measure is defined as a difference between a depth of the three-dimensional point in the three-dimensional model and a depth of the associated three-dimensional point in the current three-dimensional model.

15. The system according to claim 12 , wherein a refined depth of the three-dimensional point is determined from the depth of the three-dimensional point in the three-dimensional model and the depth of the associated three-dimensional point in the current three-dimensional model and is used for updating the depth of the three-dimensional point in at least one of the current three-dimensional model and in the three dimensional model.

16. The system according to claim 1 , wherein at least one of the current depth samples is discarded if it is determined that the depth value is above a defined threshold.

17. The system according to claim 1 , wherein the image registration method is feature-based or intensity-based.

18. The system according to claim 17 , wherein in the feature-based image registration method, two-dimensional image features from the set of reference two-dimensional imaged points are extracted and the camera motion is estimated by determining two-dimensional correspondences based on the extracted image features.

19. The system according to claim 1 , wherein at least one of the three-dimensional model and current three-dimensional model further comprise additional information which is determined from at least one of the set of reference two-dimensional imaged points and the set of current two-dimensional imaged points using the intrinsic parameters.

20. The system according to claim 19 , wherein the additional information is determined by associating the color of at least a corresponding reference two-dimensional imaged point to the three-dimensional model.

21. A non-transitory computer readable medium comprising software code sections which are adapted to

obtain a set of reference two-dimensional imaged points captured by a camera at a first camera pose and reference depth samples associated to the set of reference two-dimensional imaged points;

determine a three-dimensional model of the environment using the reference depth samples and intrinsic parameters of the camera;

provide a set of current two-dimensional imaged points captured by the camera at a second camera pose and current depth samples associated to the set of current two-dimensional imaged points and determining a current three-dimensional model using the current depth samples and the intrinsic parameters;

estimate a camera motion between the first camera pose and the second camera pose by means of an image registration method;

determine based on the estimated camera motion a similarity measure between the three-dimensional model and the current three-dimensional model;

in response to determining that the similarity measure meets a first condition, updating the three-dimensional model of the environment using at least one point of the current three-dimensional model; and

in response to updating the three-dimensional model using at least one point of the current three-dimensional model, adding the set of current two-dimensional imaged points to the set of reference two-dimensional imaged points.