Methods and systems for color point cloud generation

1. A method for generating color point cloud, implemented on a computing device having a processor, at least one non-transitory computer-readable storage medium, and a communication platform connected to a network, the method comprising:

receiving a point cloud and a plurality of images with respect to a scene captured by a plurality of sensors associated with a vehicle as the vehicle moves along a trajectory;

segmenting, by the processor, the point cloud into a plurality of segments each associated with a start point and an end point on the trajectory of the vehicle;

associating, by the processor, each segment of the point cloud with one or more of the plurality of images based on the start point and the end point; and

generate color point cloud, by the processor, by aggregating each segment of the point cloud and the one or more of the plurality of images based on calibration parameter in different distances between the segment of the point cloud and the vehicle.

2. The method of claim 1 , wherein each segment of the point cloud is associated with a same moving distance of the vehicle on the trajectory between the start point and the end point.

3. The method of claim 1 , wherein segmenting the point cloud is based on time, position, and orientation of the vehicle on the trajectory.

4. The method of claim 1 , wherein:

associating the segment of the point cloud with the one or more of the plurality of images comprises:

determining a first image of the plurality of images based on the start point and a first threshold, wherein a first distance between the start point and a first base point where the first image is captured equals the first threshold, and

determining a second image of the plurality of images based on the end point and a second threshold, wherein a second distance between the end point and a second base point where the second image is captured equals the second threshold; and

the one or more of the plurality of images include images captured between the first base point and the second base point on the trajectory.

5. The method of claim 4 , wherein the first base point is ahead of the start point on the trajectory, and the second base point is ahead of the end point on the trajectory.

6. The method of claim 1 , further comprising:

determining a plurality sets of calibration parameters associated with each segment of the point cloud based on the distances between the segment of the point cloud and the vehicle; and

aggregating the segment of the point cloud and the one or more of the plurality of images based on the calibration parameter.

7. The method of claim 6 , wherein determining a plurality of sets of calibration parameters associated with the segment of the point cloud comprises:

dividing the point cloud into a plurality of groups, each group of the point cloud associated with a distance between the respective group of the point cloud and the vehicle;

determining calibration parameters for each group of the point cloud based on a point in the group of the point cloud; and

assigning, to the segment of the point cloud, the calibration parameter for the group of the point cloud to which the segment of the point cloud belongs.

8. The method of claim 6 , wherein aggregating the segment of the point cloud and the one or more of the plurality of images comprises:

selecting some of the one or more of the plurality of images;

for each of the selected images, matching a plurality of pixels in the image to each point in the segment of the point cloud based on the calibration parameter; and

assigning color information associated with each of the plurality of pixels to the respective point in the segment of the point cloud.

9. The method of claim 1 , wherein the one or more of the plurality of images include a front view image of the scene.

10. A system for generating color point cloud, comprising:

a communication interface configured to receive a point cloud and a plurality of images with respect to a scene captured by a plurality of sensors equipped on a vehicle as the vehicle moves along a trajectory;

a storage configured to store the point cloud and the plurality of images; and

a processor configured to:

segment the point cloud into a plurality of segments each associated with a start point and an end point on the trajectory of the vehicle;

associate each segment of the point cloud with one or more of the plurality of images based on the start point and the end point; and

generate color point cloud by aggregating each segment of the point cloud and the one or more of the plurality of images based on calibration parameter in different distances between the segment of the point cloud and the vehicle.

11. The system of claim 10 , wherein each segment of the point cloud is associated with a same moving distance of the vehicle on the trajectory between the start point and the end point.

12. The system of claim 10 , wherein segmenting the point cloud is based on time, position, and orientation of the vehicle on the trajectory.

13. The system of claim 10 , wherein:

to associate the segment of the point cloud with the one or more of the plurality of images, the processor is configured to:

determine a first image of the plurality of images based on the start point and a first threshold, wherein a first distance between the start point and a first base point where the first image is captured equals the first threshold, and

determine a second image of the plurality of images based on the end point and a second threshold, wherein a second distance between the end point and a second base point where the second image is captured equals the second threshold; and

the one or more of the plurality of images include images captured between the first base point and the second base point on the trajectory.

14. The system of claim 13 , wherein the first base point is ahead of the start point on the trajectory, and the second base point is ahead of the end point on the trajectory.

15. The system of claim 10 , wherein the processor is further configured to:

determine a plurality of sets of calibration parameters associated with each segment of the point cloud based on the distance between the segment of the point cloud and the vehicle; and

aggregate the segment of the point cloud and the one or more of the plurality of images based on the calibration parameter.

16. The system of claim 15 , wherein to determine the plurality of sets of calibration parameters associated with the segment of the point cloud, the processor is configured to:

divide the point cloud into a plurality of groups, each group of the point cloud associated with a distance between the respective group of the point cloud and the vehicle;

determine calibration parameters for each group of the point cloud based on a point in the group of the point cloud; and

assign, to the segment of the point cloud, the calibration parameter for the group of the point cloud to which the segment of the point cloud belongs.

17. The system of claim 15 , wherein to aggregate the segment of the point cloud and the one or more of the plurality of images, the processor is configured to:

select some of the one or more of the plurality of images;

for each of the selected images, match a plurality of pixels in the image to each point in the segment of the point cloud based on the calibration parameter; and

assign color information associated with each of the plurality of pixels to the respective point in the segment of the point cloud.

18. The system of claim 10 , wherein the one or more of the plurality of images include a front view image of the scene.

19. The system of claim 10 , wherein the plurality of sensors include a Light Detection And Ranging (LiDAR) laser scanner and one or more monocular cameras.

20. A non-transitory computer-readable medium having instructions stored thereon that, when executed by one or more processors, causes the one or more processors to perform operations comprising:

receiving a point cloud and a plurality of images with respect to a scene captured by a plurality of sensors associated with a vehicle as the vehicle moves along a trajectory;

segmenting the point cloud into a plurality of segments each associated with a start point and an end point on the trajectory of the vehicle;

associating each segment of the point cloud with one or more of the plurality of images based on the start point and the end point; and

generating color point cloud by aggregating each segment of the point cloud and the one or more of the plurality of images based on calibration parameter in different distances between the segment of the point cloud and the vehicle.