Granted Patent B2

US 12,266,074 · App. 17/758,692 · Granted Apr 1, 2025

Inventors: Deguo Xia (Beijing, CN); Jizhou Huang (Beijing, CN); Haifeng Wang (Beijing, CN)

Assignee: NETCOM SCIENCE TECHNOLOGY CO., LTD.

G06T3/14G06T5/80G06T7/38G06T7/70

Abstract

Claims (64)

1. A method for generating a high definition map, comprising:

acquiring point cloud data and front-view image data which are collected respectively by a plurality of collecting devices at a plurality of location points, to obtain a sequence of point clouds and a sequence of front-view images;

performing registration of the front-view images and the point clouds on the sequence of point clouds and the sequence of front-view images, wherein adjacent front-view images are registered to obtain a set of registered front-view images, the point clouds of the sequence of point clouds are registered on a reference point cloud, and the registered point clouds are projected onto the obtained set of registered front-view images, so as to obtain coordinate information of pixels in the front-view images;

transforming the sequence of front-view images into a top-view image based on the result of the registration and determining coordinate information of pixels in the top-view image, comprising: transforming respectively the front-view images in the sequence of front-view images into top-view images based on inverse perspective transformation; performing matching on corresponding top-view images according to the coordinate information of pixels in the front-view images, so as to determine the coordinate information of pixels in the top-view images; and splicing the top-view images according to the coordinate information of the pixels in the top-view images, so as to obtain finally the top-view image; and

identifying map elements of the top-view image to obtain the high definition map.

2. The method according to claim 1 , wherein performing registration of the front-view images and the point clouds on the sequence of point clouds and the sequence of front-view images comprises:

registering the adjacent images in the sequence of front-view images to obtain sets consisting of corresponding pixels in the adjacent images; and

projecting the registered point cloud data into the sets, so as to obtain coordinate information of pixels in the sets.

3. The method according to claim 2 , further comprising: before projecting the point cloud data into the sets,

performing distortion correction on the point cloud data according to a movement amount of 360 degrees of a laser radar device collecting the point cloud data.

4. The method according to claim 2 , further comprising: before projecting the point cloud data into the sets,

determining the reference point cloud in the sequence of point clouds; and

registering other point clouds one by one by taking the reference point cloud as a reference.

5. The method according to claim 4 , wherein determining the reference point cloud in the sequence of point clouds comprises:

taking a first point cloud in the sequence of point clouds as a reference, and registering other point clouds one by one; and

taking a point cloud in the sequence of point clouds having highest proportion of matching points with a previous point cloud and a next point cloud as the reference point cloud.

6. The method according to claim 4 , wherein registering other point clouds one by one comprises:

learning a transformation matrix between the reference point cloud and an adjacent point cloud not registered from points in the two point clouds;

transforming the reference point cloud using the transformation matrix to obtain the adjacent point cloud registered; and

taking the adjacent point cloud as a new reference, and proceeding to perform the step of learning the transformation matrix between the reference point cloud and the adjacent point cloud not registered from points in the two point clouds, until all point cloud data in the sequence of point clouds is registered.

7. The method according to claim 6 , wherein the learning the transformation matrix between the reference point cloud and the adjacent point cloud not registered from points in the two point clouds comprises:

learning the transformation matrix between the reference point cloud and the adjacent point cloud from the points in the two point clouds using an ICP algorithm;

wherein the ICP algorithm uses a loss function of: a distance mean or weighted mean between each transformed point obtained by transforming each point in the reference point cloud according to the transformation matrix and each point in the adjacent point cloud closest to the each transformed point.

8. The method according to claim 7 , further comprising: before determining the reference point cloud in the sequence of point clouds,

determining a set consisting of corresponding points in the adjacent images; and

when determining the weighted mean, determining a weight for each distance according to whether a point in the reference point cloud belongs to the set consisting of the corresponding points.

9. The method according to claim 2 , wherein projecting the point cloud data into the sets, so as to obtain coordinate information of each pixel in the sets comprises:

projecting coordinates of the point cloud data to the sets to obtain the coordinate information of points corresponding to pixels in the front-view images; and

transforming the coordinate information of the points corresponding to the pixels in the front-view images into the coordinate information of the pixels according to a transformation and translation matrix from a laser radar coordinate system to an image-collecting-device coordinate system.

10. The method according to claim 1 , wherein identifying the map elements of the top-view image to obtain the high definition map comprises:

identifying road information of the top-view image; and

overlaying the road information identified onto the top-view image for display, to obtain the high definition map.

11. An electronic device, comprising:

at least one processor; and

a memory connected with the at least one processor communicatively;

wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method for generating a high definition map, which comprises:

acquiring point cloud data and front-view image data which are collected respectively by a plurality of collecting devices at a plurality of location points, to obtain a sequence of point clouds and a sequence of front-view images;

performing registration of the front-view images and the point clouds on the sequence of point clouds and the sequence of front-view images, wherein adjacent front-view images are registered to obtain a set of registered front-view images, the point clouds of the sequence of point clouds are registered on a reference pint cloud, and the registered point clouds are projected onto the obtained set of registered front-view images, so as to obtain coordinate information of pixels in the front-view images;

transforming the sequence of front-view images into a top-view image based on the result of the registration and determining coordinate information of pixels in the top-view image, comprising: transforming respectively the front-view images in the sequence of front-view images into top-view images based on inverse perspective transformation; performing matching on corresponding top-view images according to the coordinate information of pixels in the front-view images, so as to determine the coordinate information of pixels in the top-view images; and splicing the top-view images according to the coordinate information of the pixels in the top-view images, so as to obtain finally the top-view image; and

identifying map elements of the top-view image to obtain the high definition map.

12. The electronic device according to claim 11 , wherein performing registration of the front-view images and the point clouds on the sequence of point clouds and the sequence of front-view images comprises:

registering the adjacent images in the sequence of front-view images to obtain sets consisting of corresponding pixels in the adjacent images; and

projecting the registered point cloud data into the sets, so as to obtain coordinate information of pixels in the sets.

13. The electronic device according to claim 12 , wherein the method further comprises: before projecting the point cloud data into the sets,

performing distortion correction on the point cloud data according to a movement amount of 360 degrees of a laser radar device collecting the point cloud data.

14. The electronic device according to claim 12 , wherein the method further comprises: before projecting the point cloud data into the sets,

determining the reference point cloud in the sequence of point clouds; and

registering other point clouds one by one by taking the reference point cloud as a reference.

15. The electronic device according to claim 12 , wherein projecting the point cloud data into the sets, so as to obtain coordinate information of each pixel in the sets comprises:

projecting coordinates of the point cloud data to the sets to obtain the coordinate information of points corresponding to pixels in the front-view images; and

transforming the coordinate information of the points corresponding to the pixels in the front-view images into the coordinate information of the pixels according to a transformation and translation matrix from a laser radar coordinate system to an image-collecting-device coordinate system.

16. A non-transitory computer readable storage medium with computer instructions stored thereon, wherein the computer instructions are used for causing a computer to perform a method for generating a high definition map, which comprises:

acquiring point cloud data and front-view image data which are collected respectively by a plurality of collecting devices at a plurality of location points, to obtain a sequence of point clouds and a sequence of front-view images;

performing registration of the front-view images and the point clouds on the sequence of point clouds and the sequence of front-view images, wherein adjacent front-view images are registered to obtain a set of registered front-view images, the point clouds of the sequence of point clouds are registered on a reference pint cloud, and the registered point clouds are projected onto the obtained set of registered front-view images, so as to obtain coordinate information of pixels in the front-view images;

transforming the sequence of front-view images into a top-view image based on the result of the registration and determining coordinate information of pixels in the top-view image, comprising: transforming respectively the front-view images in the sequence of front-view images into top-view images based on inverse perspective transformation; performing matching on corresponding top-view images according to the coordinate information of pixels in the front-view images, so as to determine the coordinate information of pixels in the top-view images; and splicing the top-view images according to the coordinate information of the pixels in the top-view images, so as to obtain finally the top-view image; and

identifying map elements of the top-view image to obtain the high definition map.

17. The non-transitory computer readable storage medium according to claim 16 , wherein performing registration of the front-view images and the point clouds on the sequence of point clouds and the sequence of front-view images comprises:

registering the adjacent images in the sequence of front-view images to obtain sets consisting of corresponding pixels in the adjacent images; and

projecting the registered point cloud data into the sets, so as to obtain coordinate information of pixels in the sets.

18. The non-transitory computer readable storage medium according to claim 17 , wherein the method further comprises: before projecting the point cloud data into the sets,

performing distortion correction on the point cloud data according to a movement amount of 360 degrees of a laser radar device collecting the point cloud data.

19. The non-transitory computer readable storage medium according to claim 17 , wherein the method further comprises: before projecting the point cloud data into the sets,

determining the reference point cloud in the sequence of point clouds; and

registering other point clouds one by one by taking the reference point cloud as a reference.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (50)