Classification of surfaces as hard/soft for combining data captured by autonomous vehicles for generating high definition maps
A high-definition map system receives sensor data from vehicles travelling along routes and combines the data to generate a high definition map for use in driving vehicles, for example, for guiding autonomous vehicles. A pose graph is built from the collected data, each pose representing location and orientation of a vehicle. The pose graph is optimized to minimize constraints between poses. Points associated with surface are assigned a confidence measure determined using a measure of hardness/softness of the surface. A machine-learning-based result filter detects bad alignment results and prevents them from being entered in the subsequent global pose optimization. The alignment framework is parallelizable for execution using a parallel/distributed architecture. Alignment hot spots are detected for further verification and improvement. The system supports incremental updates, thereby allowing refinements of subgraphs for incrementally improving the high-definition map for keeping it up to date.
1. A method for generating high definition maps based on classification of surfaces, for use in driving of autonomous vehicles, the method comprising:
generating a first three dimensional representation of a geographical region based on sensor data captured by autonomous vehicles driving through the geographical region;
generating a second three dimensional representation of a geographical region based on sensor data captured by autonomous vehicles driving through the geographical region;
for each of a plurality of points in each three dimensional (3D) representation:
identifying a surface of a structure associated with the point;
determining a measure of hardness of the surface based on one or more criteria; and
determining a measure of confidence for the point based on the determined measure of hardness of the surface of the structure;
determining a transformation to map the first 3D representation to the second 3D representation based on iterative closest point (ICP) method by weighing data points based on the measure of confidence for each point;
determining a high definition map of the region by combining the first 3D representation to the second 3D representation using the transformation; and
providing the high definition map for driving one or more autonomous vehicles.
2. The method of claim 1 , wherein determining the transformation to map the first 3D representation to the second 3D representation comprises, weighing corresponding points higher if the points have surfaces with matching surface type compared to corresponding points that have different surface types, wherein a surface type is one of hard or soft.
3. The method of claim 1 , determining the transformation to map the first 3D representation to the second 3D representation comprises, weighing corresponding points higher if the corresponding points are on surfaces with matching measure of hardness, wherein a first surface has a matching measure of hardness with a second surface if the measure of hardness of the first surface is within a threshold value of the measure of hardness of the second surface.
4. The method of claim 1 , determining the transformation to map the first 3D representation to the second 3D representation comprises, weighing corresponding points higher if the corresponding points are both on surfaces with a hard surface type compared to corresponding points that are both on surfaces with a soft surface type.
5. The method of claim 1 , wherein determining a measure of hardness for the surface based on one or more criteria comprises:
determining a measure of variance of normal vectors of a plurality of points on the surface of the structure, wherein the plurality of points are in the neighborhood of the point.
6. The method of claim 5 , wherein the measure of confidence is a value inversely related to the measure of variance of the normal vectors of the plurality of points on the surface of the structure.
7. The method of claim 1 , wherein the one or more criteria for determining the measure of hardness for the surface comprises a color or surface determined from an image of the structure captured by a camera mounted on an autonomous vehicle.
8. The method of claim 1 , wherein the one or more criteria for determining the measure of hardness for the surface comprises a number of LIDAR laser returns associated with the surface of the structure.
9. The method of claim 1 , wherein the one or more criteria for determining the measure of hardness for the surface comprises an intensity of the LIDAR laser associated with the surface of the structure.
10. The method of claim 1 , wherein the three dimensional representation of the geographical region is a point cloud representation.
11. The method of claim 1 , wherein the sensor data captured by a vehicle comprises LIDAR range images captured by a LIDAR mounted on the vehicle.
12. The method of claim 11 , wherein determining a normal vector for point in a LIDAR range image comprises:
determining a plurality of neighbor points for the point in the LiDAR range image;
determining a horizontal vector and a vertical vector between the point and the neighbor points; and
determining the normal vector as a cross product of the horizontal vector and the normal vector.
13. The method of claim 11 , wherein determining the measure of confidence of a point comprises:
selecting a window in the LIDAR range image;
selecting a plurality of neighbor points for the point in the selected window in the LiDAR range image;
determining a variance value as an aggregate of distances between the point and the neighbor points; and
determining the measure of confidence based on the variance.
14. A non-transitory computer readable storage medium storing instructions for:
generating a first three dimensional representation of a geographical region based on sensor data captured by autonomous vehicles driving through the geographical region;
generating a second three dimensional representation of a geographical region based on sensor data captured by autonomous vehicles driving through the geographical region;
for each of a plurality of points in each three dimensional (3D) representation:
identifying a surface of a structure associated with the point;
determining a measure of hardness of the surface based on one or more criteria; and
determining a measure of confidence for the point based on the determined measure of hardness of the surface of the structure;
determining a transformation to map the first 3D representation to the second 3D representation based on iterative closest point (ICP) method by weighing data points based on the measure of confidence for each point;
determining a high definition map of the region by combining the first 3D representation to the second 3D representation using the transformation; and
providing the high definition map for driving one or more autonomous vehicles.
15. The non-transitory computer readable storage medium of claim 14 , wherein the instructions for determining a measure of confidence for a point comprise instructions for:
determining a measure of variance of normal vectors of a plurality of points on the surface of the structure, wherein the plurality of points are in the neighborhood of the point.
16. The non-transitory computer readable storage medium of claim 15 , wherein the measure of confidence is a value inversely related to the measure of variance of the normal vectors of the plurality of points on the surface of the structure.
17. The non-transitory computer readable storage medium of claim 14 , wherein the sensor data captured by a vehicle comprises LIDAR range images captured by a LIDAR mounted on the vehicle.
18. The non-transitory computer readable storage medium of claim 14 , wherein the instructions for determining a normal vector for point in a LIDAR range image comprise instructions for:
determining a plurality of neighbor points for the point in the LiDAR range image;
determining a horizontal vector and a vertical vector between the point and the neighbor points; and
determining the normal vector as a cross product of the horizontal vector and the normal vector.
19. The non-transitory computer readable storage medium of claim 14 , wherein the instructions for determining the measure of confidence of a point comprise instructions for:
selecting a window in the LIDAR range image;
selecting a plurality of neighbor points for the point in the selected window in the LiDAR range image;
determining a variance value as an aggregate of distances between the point and the neighbor points; and
determining the measure of confidence based on the variance.
20. A computer system comprising:
an electronic processor; and
a non-transitory computer readable storage medium storing instructions executable by the electronic processor, the instructions for:
generating a first three dimensional representation of a geographical region based on sensor data captured by autonomous vehicles driving through the geographical region;
generating a second three dimensional representation of a geographical region based on sensor data captured by autonomous vehicles driving through the geographical region;
for each of a plurality of points in each three dimensional (3D) representation:
identifying a surface of a structure associated with the point;
determining a measure of hardness of the surface based on one or more criteria; and
determining a measure of confidence for the point based on the determined measure of hardness of the surface of the structure;
determining a transformation to map the first 3D representation to the second 3D representation based on iterative closest point (ICP) method by weighing data points based on the measure of confidence for each point;
determining a high definition map of the region by combining the first 3D representation to the second 3D representation using the transformation; and
providing the high definition map for driving one or more autonomous vehicles.
21. The computer system of claim 20 , wherein the instructions for determining a measure of confidence for a point comprise instructions for:
determining a measure of variance of normal vectors of a plurality of points on the surface of the structure, wherein the plurality of points are in the neighborhood of the point.
22. The computer system of claim 21 , wherein the measure of confidence is a value inversely related to the measure of variance of the normal vectors of the plurality of points on the surface of the structure.
23. The computer system of claim 20 , wherein the sensor data captured by a vehicle comprises LIDAR range images captured by a LIDAR mounted on the vehicle.
24. The computer system of claim 20 , wherein the instructions for determining a normal vector for point in a LIDAR range image comprise instructions for:
determining a plurality of neighbor points for the point in the LiDAR range image;
determining a horizontal vector and a vertical vector between the point and the neighbor points; and
determining the normal vector as a cross product of the horizontal vector and the normal vector.
25. The computer system of claim 20 , wherein the instructions for determining the measure of confidence of a point comprise instructions for:
selecting a window in the LIDAR range image;
selecting a plurality of neighbor points for the point in the selected window in the LiDAR range image;
determining a variance value as an aggregate of distances between the point and the neighbor points; and
determining the measure of confidence based on the variance.