3D submap reconstruction system and method for centimeter precision localization using camera-based submap and LiDAR-based global map

1. A method for localization of an autonomous vehicle, comprising:

receiving images from a camera, generating a 3D submap from the images, and converting the 3D submap into first voxels;

receiving data from a light detection and ranging (LiDAR), generating a global map from the data, and converting the global map into second voxels;

estimating a distribution of 3D points within the first and second voxels using a probabilistic model;

extracting features from the 3D submap and the global map;

classifying the extracted features into classes; and

matching the features extracted from the 3D submap against features extracted from the global map.

2. The method according to claim 1 , further comprising:

constructing the 3D submap and the global map before the voxelizing the 3D submap and the global map into voxels.

3. The method according to claim 2 , further comprising:

performing data alignment before constructing the 3D submap and the global map; and

collecting the data in an environment by using sensors including the camera, the LiDAR, and an inertial navigation module.

4. The method according to claim 2 , wherein the constructing the 3D submap comprises:

constructing at least one 3D submap based on the images using visual SLAM.

5. The method according to claim 2 , wherein the constructing the global map comprises:

constructing a city-scale 3D map based on the data from the LiDAR using LiDAR mapping.

6. The method according to claim 1 , wherein the extracting the features from the 3D submap and the global map comprises:

extracting structured features and unstructured features from the 3D submap and the global map.

7. The method according to claim 6 , wherein the structured features include at least one of planes, straight lines and curved lines, and the unstructured features include sparse 3D points.

8. The method according to claim 1 , wherein the matching the features comprises:

computing matching scores between corresponding features of a same class in the 3D submap and the global map; and

selecting, for each feature in the 3D submap, a corresponding feature with a highest matching score from the global map.

9. The method according to claim 1 , further comprising:

refining a feature correspondence; and

refining a location of the 3D submap.

10. The method according to claim 9 , wherein the refining the feature correspondence comprises:

determining that a distance between corresponding features is larger than a threshold; and

removing feature correspondence associated with the corresponding features.

11. The method according to claim 10 , wherein the distance between the corresponding features is determined by a trained classifier.

12. The method according to claim 9 , wherein the refining the location of the 3D submap comprises:

performing an iterative estimation of the location of the 3D submap until the distance between corresponding features reaches a predetermined value.

13. A system for generating a global map, the system comprising:

a memory;

one or more processors; and

one or more programs stored in the memory that when executed by the one or more processors perform at least:

obtaining data from a light detection and ranging (LiDAR) sensor, images from a camera, and location information from an inertial navigation module;

performing data alignment of the data, the images, and the location information;

removing distortion from a motion of points in the LiDAR data;

estimating positions of the points in the LiDAR data using interpolation;

generating a global map using the positions of the points; and

increasing precision of the global map using a point registration.

14. The system according to claim 13 , wherein the location information is obtained from a global navigation satellite system (GNSS)-inertial system.

15. The system according to claim 13 , wherein the point registration uses an iterative closest point (ICP) method.

16. The system according to claim 15 , wherein the global map is a 3D city city-scale map.

17. A method for generating a global map, comprising:

obtaining data from a light detection and ranging (LiDAR) sensor, images from a camera, and location information from an inertial navigation module;

performing data alignment of the data, the images, and the location information;

removing distortion from a motion of points in the LiDAR data;

estimating positions of the points in the LiDAR data using interpolation;

generating a global map using the positions of the points; and

increasing precision of the global map using a point registration.

18. The method according to claim 17 , wherein the location information is obtained from a global navigation satellite system (GNSS)-inertial system.

19. The method according to claim 17 , wherein the point registration is an iterative closest point (ICP) method.

20. The method according to claim 17 , wherein the global map is a 3D city-scale map.