Feature extraction from 3D submap and global map system and method for centimeter precision localization using camera-based submap and lidar-based global map
A method of localization for a non-transitory computer readable storage medium storing one or more programs is disclosed. The one or more programs comprise instructions, which when executed by a computing device, cause the computing device to perform utilizing one or more autonomous vehicle driving modules that execute processing of images from a camera and data from a LiDAR the following steps comprising: aligning a 3D submap with a global map; extracting features from the 3D submap and the global map; classifying the extracted features in classes; and establishing correspondence of features in a same class between the 3D submap and the global map.
1. A method of localization for a non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing device, cause the computing device to perform by one or more autonomous vehicle driving modules execution of processing of images from a camera and data from a LiDAR using the following steps comprising:
constructing a 3D submap based on the images from the camera;
constructing a global map based on the data from the LiDAR, wherein the camera and the LiDAR are with a same vehicle;
aligning the 3D submap with the global map;
extracting features from the 3D submap and the global map;
classifying the extracted features in classes; and
establishing correspondence of features in a same class between the 3D submap and the global map.
2. The method according to claim 1 , wherein constructing the 3D submap comprises:
obtaining the images from the camera; and
constructing the 3D submap based on the images, using visual SLAM.
3. The method according to claim 1 , wherein constructing a global map comprises:
obtaining the data from the LiDAR; and
constructing a city-scale 3D map based on the data from the LiDAR, using LiDAR mapping.
4. The method according to claim 1 , wherein aligning the 3D submap with the global map further comprises:
transforming coordinate of the 3D submap into coordinate of the global map.
5. The method according to claim 1 , wherein extracting features from the 3D submap and the global map comprises:
extracting structured features and unstructured features from 3D submap and the global map.
6. The method according to claim 5 , wherein the structured features include at least one of planes, straight lines and curved lines, and the unstructured features include sparse 3D points.
7. The method according to claim 1 , wherein extracting features from the 3D submap and the global map comprises:
voxelizing the 3D submap and the global map into voxels; and
estimating distribution of 3D points within the voxels, using a probabilistic model.
8. The method according to claim 1 , wherein establishing correspondence of features comprises:
computing matching scores between corresponding features of a same class in the 3D submap and the global map; and
selecting, for each feature in the 3D submap, a corresponding feature with the highest matching score from the global map.
9. The method according to claim 1 further comprising:
determining that a distance between corresponding features is larger than a threshold; and
removing, based on the alignment of the 3D submap with the global map, feature correspondence associated with the corresponding features.
10. The method according to claim 9 , wherein the distance between corresponding features is determined by a trained classifier.
11. The method according to claim 1 further comprising:
performing an iterative estimation of location of the 3D submap until distance between corresponding features reaches a predetermined value.
12. A system for localization, the system comprising:
an internet server, comprising:
an I/O port, configured to transmit and receive electrical signals to and from a client device;
a memory;
one or more processing units; and
one or more programs stored in the memory and configured for execution by the one or more processing units, the one or more programs including instructions utilizing one or more autonomous vehicle driving modules that execute processing of images from a camera and data from a LiDAR for:
constructing a 3D submap based on the images from the camera;
constructing a global map based on the data from the LiDAR, wherein the camera and the LiDAR are with a same vehicle;
aligning the 3D submap with the global map;
extracting features from the 3D submap and the global map;
classifying the extracted features in classes; and
establishing correspondence of features in a same class between the 3D submap and the global map.
13. The system according to claim 12 , wherein aligning the 3D submap with the global map further comprises:
transforming coordinate of the 3D submap into coordinate of the global map.
14. The system according to claim 12 , wherein extracting features from the 3D submap and the global map comprises:
extracting structured features and unstructured features from 3D submap and the global map.
15. The system according to claim 14 , wherein the structured features include at least one of planes, straight lines and curved lines, and the unstructured features include sparse 3D points.
16. The system according to claim 12 , wherein extracting features from the 3D submap and the global map comprises:
voxelizing the 3D submap and the global map into voxels; and
estimating distribution of 3D points within the voxels, using a probabilistic model.
17. The system according to claim 12 , wherein establishing correspondence of features comprises:
computing matching scores between corresponding features of a same class in the 3D submap and the global map; and
selecting, for each feature in the 3D submap, a corresponding feature with the highest matching score from the global map.
18. The system according to claim 12 further comprising:
determining that a distance between corresponding features is larger than a threshold; and
removing, based on the alignment of the 3D submap with the global map, feature correspondence associated with the corresponding features.
19. The system according to claim 12 further comprising:
performing an iterative estimation of location of the 3D submap until distance between corresponding features reaches a predetermined value.