Point cloud alignment systems for generating high definition maps for vehicle navigation
A method for determining a trajectory of a vehicle within a physical space based at least on an aligned point cloud may include generating a fused feature map by concatenating a first feature map corresponding to a source point cloud and a second feature map corresponding to a target point cloud. A machine learning model may be applied to determine, based at least on the fused feature map, a relative transform aligning the target point cloud to the source point cloud. An aligned target point cloud may be generated by transforming the target point cloud in accordance with the relative transform. Furthermore, a trajectory of a vehicle within the physical space may be determined based on at least the first relative transform. Related systems and computer program products are also provided.
1. A method, comprising:
generating, using at least one data processor, a fused feature map by at least concatenating a first feature map corresponding to a source point cloud and a second feature map corresponding to a target point cloud, the target point cloud corresponding to at least a portion of a same physical space as the source point cloud;
applying, using the at least one data processor, a machine learning model trained to determine, based at least on the fused feature map, a first relative transform aligning the target point cloud to the source point cloud, wherein the machine learning model determines the first relative transform by at least performing a weighted subsampling to extract, from the fused feature map, corresponding features from the source point cloud and the target point cloud such that the first feature map and the second feature map are downsampled into a combined feature map;
generating, using the at least one data processor, an aligned target point cloud by at least transforming the target point cloud in accordance with the first relative transform; and
determining, using the at least one data processor, a trajectory of a vehicle within the physical space based on at least the first relative transform.
2. The method of claim 1 , wherein the first feature map corresponds to an encoded representation of the source point cloud, and wherein the second feature map corresponds to an encoded representation of the target point cloud.
3. The method of claim 2 , further comprising:
encoding, using the at least one data processor, the source point cloud and the target point cloud such that one or more objects present in each point cloud are represented as one or more pillars, each of the one or more pillars bounding a plurality of points associated with a corresponding object.
4. The method of claim 1 , further comprising:
pre-training, using the at least one data processor, the machine learning model prior to training the machine learning model to determine the first relative transform, the pre-training includes reconstructing, using the at least one data processor, a point cloud by at least decoding an encoding of the point cloud deformed by one or more offsets determined by the machine learning model, and adjusting the machine learning model to minimize a difference between the point cloud and the decoded point cloud.
5. The method of claim 4 , further comprising:
pre-training, based at least on the difference between the point cloud and the decoded point cloud, an encoder generating the encoding of the point cloud.
6. The method of claim 1 , wherein the machine learning model includes a 1×1 convolution layer configured to perform the weighted subsampling.
7. The method of claim 1 , wherein the machine learning model further determines the first relative transform by at least flattening the combined feature map into a one-dimensional vector for processing by a fully connected layer of the machine learning model.
8. The method of claim 1 , further comprising:
applying, using the at least one data processor, the first relative transform to perform a coarse point registration of the target point cloud;
applying, using the at least one data processor, the machine learning model trained to determine a second relative transform to further align the aligned target point cloud to the source point cloud; and
applying, using the at least one data processor, the second relative transform to perform a fine point registration of the target point cloud.
9. The method of claim 1 , wherein the first relative transform includes a translation along at least one of an x-axis, a y-axis, and a z-axis.
10. The method of claim 1 , wherein the first relative transform includes a rotation around a fixed point of θ radians about a unit axis (X, Y, Z).
11. The method of claim 1 , wherein the source point cloud and the target point cloud comprise three dimensional point clouds.
12. A system, comprising:
at least one data processor, and
at least one non-transitory storage media storing instructions that, when executed by the at least one processor, cause the at least one processor to perform operations comprising:
generating, using the at least one data processor, a fused feature map by at least concatenating a first feature map corresponding to a source point cloud and a second feature map corresponding to a target point cloud, the target point cloud corresponding to at least a portion of a same physical space as the source point cloud;
applying, using the at least one data processor, a machine learning model trained to determine, based at least on the fused feature map, a first relative transform aligning the target point cloud to the source point cloud, wherein the machine learning model determines the first relative transform by at least performing a weighted subsampling to extract, from the fused feature map, corresponding features from the source point cloud and the target point cloud such that the first feature map and the second feature map are downsampled into a combined feature map;
generating, using the at least one data processor, an aligned target point cloud by at least transforming the target point cloud in accordance with the first relative transform; and
determining, using the at least one data processor, a trajectory of a vehicle within the physical space based on at least the first relative transform.
13. The system of claim 12 , wherein the first feature map corresponds to an encoded representation of the source point cloud, and wherein the second feature map corresponds to an encoded representation of the target point cloud.
14. The system of claim 13 , wherein the operations further comprise:
encoding, using the at least one data processor, the source point cloud and the target point cloud such that one or more objects present in each point cloud are represented as one or more pillars, each of the one or more pillars bounding a plurality of points associated with a corresponding object.
15. The system of claim 12 , wherein the operations further comprise:
pre-training, using the at least one data processor, the machine learning model prior to training the machine learning model to determine the first relative transform, the pre-training includes reconstructing, using the at least one data processor, a point cloud by at least decoding an encoding of the point cloud deformed by one or more offsets determined by the machine learning model, and adjusting the machine learning model to minimize a difference between the point cloud and the decoded point cloud.
16. The system of claim 15 , wherein the operations further comprise:
pre-training, based at least on the difference between the point cloud and the decoded point cloud, an encoder generating the encoding of the point cloud.
17. The system of claim 12 , wherein the machine learning model includes a 1×1 convolution layer configured to perform the weighted subsampling.
18. The system of claim 12 , wherein the machine learning model further determines the first relative transform by at least flattening the combined feature map into a one-dimensional vector for processing by a fully connected layer of the machine learning model.
19. The system of claim 12 , wherein the operations further comprise:
applying, using the at least one data processor, the first relative transform to perform a coarse point registration of the target point cloud;
applying, using the at least one data processor, the machine learning model trained to determine a second relative transform to further align the aligned target point cloud to the source point cloud; and
applying, using the at least one data processor, the second relative transform to perform a fine point registration of the target point cloud.
20. The system of claim 12 , wherein the first relative transform includes a translation along at least one of an x-axis, a y-axis, and a z-axis.
21. The system of claim 12 , wherein the first relative transform includes a rotation around a fixed point of θ radians about a unit axis (X, Y, Z).
22. The system of claim 12 , wherein the source point cloud and the target point cloud comprise three dimensional point clouds.
23. At least one non-transitory storage media storing instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising:
generating, using the at least one data processor, a fused feature map by at least concatenating a first feature map corresponding to a source point cloud and a second feature map corresponding to a target point cloud, the target point cloud corresponding to at least a portion of a same physical space as the source point cloud;
applying, using the at least one data processor, a machine learning model trained to determine, based at least on the fused feature map, a first relative transform aligning the target point cloud to the source point cloud, wherein the machine learning model determines the first relative transform by at least performing a weighted subsampling to extract, from the fused feature map, corresponding features from the source point cloud and the target point cloud such that the first feature map and the second feature map are downsampled into a combined feature map;
generating, using the at least one data processor, an aligned target point cloud by at least transforming the target point cloud in accordance with the first relative transform; and
determining, using the at least one data processor, a trajectory of a vehicle within the physical space based on at least the first relative transform.
24. The at least one non-transitory storage media of claim 23 , wherein the first feature map corresponds to an encoded representation of the source point cloud, and wherein the second feature map corresponds to an encoded representation of the target point cloud.
25. The at least one non-transitory storage media of claim 24 , wherein the operations further comprise:
encoding, using the at least one data processor, the source point cloud and the target point cloud such that one or more objects present in each point cloud are represented as one or more pillars, each of the one or more pillars bounding a plurality of points associated with a corresponding object.
26. The at least one non-transitory storage media of claim 23 , wherein the operations further comprise:
pre-training, using the at least one data processor, the machine learning model prior to training the machine learning model to determine the first relative transform, the pre-training including reconstructing, using the at least one data processor, a point cloud by at least decoding an encoding of the point cloud deformed by one or more offsets determined by the machine learning model, and adjusting the machine learning model to minimize a difference between the point cloud and the decoded point cloud; and
pre-training, based at least on the difference between the point cloud and the decoded point cloud, an encoder generating the encoding of the point cloud.
27. The at least one non-transitory storage media of claim 23 , wherein the machine learning model includes a 1×1 convolution layer configured to perform the weighted subsampling.