Color map layer for annotation
Some methods described include: receiving, with at least one processor, a point cloud from a pose graph; receiving, with the at least one processor, an image from a driving log of a vehicle, the image corresponding to the point cloud from the pose graph; obtaining, with the at least one processor, image pixel labels for the image; projecting, with the at least one processor, the point cloud in a point cloud coordinate system to an image coordinate system based on the image pixel labels; generating, with the at least one processor, a six-dimensional colored point cloud by combining the point cloud and color information from the image; and transforming, with the at least one processor, the six-dimensional colored point cloud into five-dimensional map tiles to form a colored map layer. Systems and computer program products are also provided.
1. A method, comprising:
receiving, with at least one processor, a point cloud from a pose graph;
receiving, with the at least one processor, an image from a driving log of a vehicle, the image corresponding to the point cloud from the pose graph;
obtaining, with the at least one processor, image pixel labels for the image;
interpolating, with the at least one processor, a pose of the image according to poses of two neighboring images, wherein the two neighboring images correspond to two point clouds, wherein the two point clouds include a first point cloud having a time stamp immediately earlier than a time stamp of the image and a second point cloud having a time stamp immediately later than the time stamp of the image;
projecting, with the at least one processor, the point cloud in a point cloud coordinate system to an image coordinate system based on the image pixel labels;
generating, with the at least one processor, a six-dimensional colored point cloud by combining the point cloud and color information from the image; and
transforming, with the at least one processor, the six-dimensional colored point cloud into five-dimensional map tiles to form a colored map layer.
2. The method of claim 1 , further comprising:
receiving calibration data from the driving log of the vehicle, wherein the calibration data includes transformation matrices including an intrinsic matrix for a camera, a rotation matrix and a translation vector; and
projecting the point cloud to the image coordinate system based on the calibration data.
3. The method of claim 1 , wherein the image pixel labels are obtained from a pre-trained image segmentation network.
4. The method of claim 1 , further comprising:
filtering out overlapping images indicating a driving distance less than a particular distance value.
5. The method of claim 1 , wherein projecting the point cloud comprises:
projecting the point cloud by a pinhole camera model.
6. The method of claim 1 , further comprising:
removing, from the point cloud, points that have different labels between the points and the corresponding image pixels.
7. The method of claim 1 , further comprising:
removing, from the point cloud, points representing dynamic objects including one or more of: pedestrians, bicycles, or vehicles.
8. The method of claim 1 , wherein projecting the point cloud further comprises:
assigning, a point color of a point in the point cloud, with a pixel color from the nearest image pixel when the point has at least two different image pixels.
9. The method of claim 1 , further comprising:
enhancing a road marker color by adding an intensity value to each point in the point cloud.
10. The method of claim 1 , further comprising:
updating the pose graph with the six-dimensional colored point cloud.
11. The method of claim 1 , wherein transforming the six-dimensional colored point cloud into the five-dimensional map tiles comprises:
removing, from the six-dimensional colored point cloud, points corresponding to an object having a height larger than a particular height value; and
transforming the six-dimensional colored point cloud into the five-dimensional map tiles by ignoring Z coordinate values.
12. A system comprising:
at least one processor; and
a memory storing instructions thereon that, when executed by the at least one processor, cause the at least one processor to perform operations, comprising:
receiving a point cloud from a pose graph;
receiving an image from a driving log of a vehicle, the image corresponding to the point cloud from the pose graph;
obtaining image pixel labels for the image;
interpolating, with the at least one processor, a pose of the image according to poses of two neighboring images, wherein the two neighboring images correspond to two point clouds, wherein the two point clouds include a first point cloud having a time stamp immediately earlier than a time stamp of the image and a second point cloud having a time stamp immediately later than the time stamp of the image;
projecting the point cloud in a point cloud coordinate system to an image coordinate system based on the image pixel labels;
generating a six-dimensional colored point cloud by combining the point cloud and color information from the image; and
transforming the six-dimensional colored point cloud into five-dimensional map tiles to form a colored map layer.
13. The system of claim 12 , the operations further comprising:
receiving calibration data from the driving log of the vehicle, wherein the calibration data includes transformation matrices including an intrinsic matrix for a camera, a rotation matrix and a translation vector; and
projecting the point cloud to the image coordinate system based on the calibration data.
14. The system of claim 12 , further comprising:
removing, from the point cloud, points that have different labels between the points and the corresponding image pixels.
15. The system of claim 12 , further comprising:
removing, from the point cloud, points representing dynamic objects including one or more of: pedestrians, bicycles, or vehicles.
16. The system of claim 12 , wherein projecting the point cloud further comprises:
assigning, a point color of a point in the point cloud, with a pixel color from the nearest image pixel when the point has at least two different image pixels.
17. The system of claim 12 , wherein transforming the six-dimensional colored point cloud into the five-dimensional map tiles comprises:
removing, from the six-dimensional colored point cloud, points corresponding to an object having a height larger than a particular height value; and
transforming the six-dimensional colored point cloud into the five-dimensional map tiles by ignoring Z coordinate values.
18. A non-transitory, computer-readable storage medium having instructions stored thereon, that when executed by at least one processor, cause the at least one processor to perform operations, comprising:
receiving a point cloud from a pose graph;
receiving an image from a driving log of a vehicle, the image corresponding to the point cloud from the pose graph;
obtaining image pixel labels for the image;
interpolating, with the at least one processor, a pose of the image according to poses of two neighboring images, wherein the two neighboring images correspond to two point clouds, wherein the two point clouds include a first point cloud having a time stamp immediately earlier than a time stamp of the image and a second point cloud having a time stamp immediately later than the time stamp of the image;
projecting the point cloud in a point cloud coordinate system to an image coordinate system based on the image pixel labels;
generating a six-dimensional colored point cloud by combining the point cloud and color information from the image; and
transforming the six-dimensional colored point cloud into five-dimensional map tiles to form a colored map layer.