Perspective conversion for multi-dimensional data analysis
Multi-dimensional data can be mapped to a projection shape and converted for image analysis. In some examples, the multi-dimensional data may include data captured by a LIDAR system for use in conjunction with a perception system for an autonomous vehicle. Converting operations can include converting three-dimensional LIDAR data to multi-channel two-dimensional data. Data points of the multi-dimensional data can be mapped to a projection shape, such as a sphere. Characteristics of the projection shape may include a shape, a field of view, a resolution, and a projection type. After data is mapped to the projection shape, the projection shape can be converted to a multi-channel, two-dimensional image. Image segmentation and classification may be performed on the two-dimensional data. Further, segmentation information may be used to segment the three-dimensional LIDAR data, while a rendering plane may be positioned relative to the segmented data to perform classification on a per-object basis.
1. A system comprising:
one or more processors; and
one or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause the one or more processors to perform operations comprising:
receiving three-dimensional data captured by a lidar sensor, the lidar sensor associated with a first perspective;
receiving segmentation information associated with the three-dimensional data;
positioning a rendering plane relative to the three-dimensional data, wherein an orientation of the rendering plane is associated with a second perspective;
projecting, as projected data and based at least in part on the segmentation information, at least a portion of the three-dimensional data onto the rendering plane; and
associating the projected data with an individual channel of a multi-channel image based at least in part on the segmentation information.
2. The system of claim 1 , wherein:
the second perspective is a top-down perspective of an environment associated with the three-dimensional data; and
the projected data is represented as two-dimensional data.
3. The system of claim 1 , wherein the segmentation information comprises a segmentation identifier associated with an object.
4. The system of claim 1 , the operations further comprising:
determining, based at least in part on the projected data, a trajectory for an autonomous vehicle; and
controlling the autonomous vehicle based at least in part on the trajectory.
5. The system of claim 1 , the operations further comprising:
inputting the multi-channel image to a machine learned model.
6. The system of claim 1 , the operations further comprising:
associating a bounding box with the projected data.
7. A method comprising:
receiving three-dimensional data captured with respect to a first perspective;
receiving segmentation information associated with the three-dimensional data;
projecting, as projected data and based at least in part on the segmentation information, at least a portion of the three-dimensional data relative to a rendering plane having an orientation associated with a second perspective; and
associating the projected data with an individual channel of a multi-channel image based at least in part on the segmentation information.
8. The method of claim 7 , wherein:
the second perspective is a top-down perspective of an environment associated with the three-dimensional data; and
the projected data is represented as two-dimensional data.
9. The method of claim 7 , wherein:
the segmentation information comprises a segmentation identifier associated with an object.
10. The method of claim 7 , further comprising:
determining, based at least in part on the projected data, a trajectory for an autonomous vehicle; and
controlling the autonomous vehicle based at least in part on the trajectory.
11. The method of claim 7 , further comprising:
inputting the multi-channel image to a machine learned model.
12. The method of claim 7 , further comprising:
associating a bounding box with the projected data.
13. One or more non-transitory computer-readable media storing instructions executable by a processor, wherein the instructions, when executed, cause the processor to perform operations comprising:
receiving three-dimensional data captured with respect to a first perspective;
receiving segmentation information associated with the three-dimensional data;
projecting, as projected data and based at least in part on the segmentation information, at least a portion of the three-dimensional data relative to a rendering plane having an orientation associated with a second perspective; and
associating the projected data with an individual channel of a multi-channel image based at least in part on the segmentation information.
14. The one or more non-transitory computer-readable media of claim 13 , wherein:
the second perspective is a top-down perspective of an environment associated with the three-dimensional data; and
the projected data is represented as two-dimensional data.
15. The one or more non-transitory computer-readable media of claim 13 , where the segmentation information comprises a segmentation identifier associated with an object.
16. The one or more non-transitory computer-readable media of claim 13 , the operations further comprising:
determining, based at least in part on the projected data, a trajectory for an autonomous vehicle; and
controlling the autonomous vehicle based at least in part on the trajectory.
17. The one or more non-transitory computer-readable media of claim 13 , the operations further comprising:
inputting the multi-channel image to a machine learned model.
18. The one or more non-transitory computer-readable media of claim 13 , the operations further comprising:
associating a bounding box with the projected data.
19. The system of claim 1 , the operations further comprising:
performing classification on the projected data to determine an object classification of an object represented in the projected data.
20. The system of claim 1 , wherein an extent of an object represented in the three-dimensional data is represented in the individual channel.