System and method for identifying travel way features for autonomous vehicle motion control
Systems and methods for identifying travel way features in real time are provided. A method can include receiving two-dimensional and three-dimensional data associated with the surrounding environment of a vehicle. The method can include providing the two-dimensional data as one or more input into a machine-learned segmentation model to output a two-dimensional segmentation. The method can include fusing the two-dimensional segmentation with the three-dimensional data to generate a three-dimensional segmentation. The method can include storing the three-dimensional segmentation in a classification database with data indicative of one or more previously generated three-dimensional segmentations. The method can include providing one or more datapoint sets from the classification database as one or more inputs into a machine-learned enhancing model to obtain an enhanced three-dimensional segmentation. And, the method can include identifying one or more travel way features based at least in part on the enhanced three-dimensional segmentation.
1. A computer-implemented method for identifying travel way features, the method comprising:
obtaining two-dimensional data and three-dimensional data associated with an environment of an autonomous vehicle at a current time;
processing the two-dimensional data with a machine-learned segmentation model to generate a two-dimensional segmentation for the environment;
generating a current three-dimensional segmentation based at least in part on the three-dimensional data and the two-dimensional segmentation;
identifying one or more historical three-dimensional datapoints previously generated for the environment at one or more times preceding the current time; and
determining one or more travel way features within the environment at the current time based on the current three-dimensional segmentation and the one or more historical three-dimensional datapoints previously generated for the environment at the one or more times preceding the current time.
2. The computer-implemented method of claim 1 , further comprising:
updating a map based on the one or more travel way features within the environment.
3. The computer-implemented method of claim 1 , further comprising:
initiating a motion of the autonomous vehicle based on the one or more travel way features within the environment.
4. The computer-implemented method of claim 1 , wherein identifying the one or more historical three-dimensional datapoints previously generated for the environment at the one or more times preceding the current time comprises:
identifying the one or more historical three-dimensional datapoints previously generated for the environment at the one or more times preceding the current time based on a spatial relationship between the one or more historical three-dimensional datapoints and the current three-dimensional segmentation.
5. The computer-implemented method of claim 1 , further comprising:
storing data indicative of the current three-dimensional segmentation in a classification database comprising the one or more historical three-dimensional datapoints previously generated for the environment at the one or more times preceding the current time.
6. The computer-implemented method of claim 5 , wherein the classification database defines a dynamic graph based on a spatial relationship between one or more of a plurality of three-dimensional points.
7. The computer-implemented method of claim 6 , wherein the current three-dimensional segmentation comprises data indicative of a plurality of current three-dimensional datapoint, and wherein the plurality of three-dimensional points comprise the plurality of current three-dimensional datapoints and the one or more historical three-dimensional datapoints.
8. The computer-implemented method of claim 1 , wherein at least one of the one or more historical three-dimensional datapoints previously generated for the environment at the one or more times preceding the current time comprises a three-dimensional point and a feature vector corresponding to the three-dimensional point.
9. The computer-implemented method of claim 8 , wherein the feature vector corresponding to the three-dimensional point comprises data indicative of at least one of a feature classification, an occlusion score, a LiDAR intensity, or a vehicle distance corresponding to the three-dimensional point.
10. The computer-implemented method of claim 9 , wherein the feature classification corresponding to the three-dimensional point comprises a class-wise probability estimate indicative of a probability that the three-dimensional point corresponds to a feature of interest.
11. An autonomous vehicle control system comprising:
one or more processors; and
one or more tangible, non-transitory, computer readable media that collectively store instructions that when executed by the one or more processors cause the autonomous vehicle control system to perform operations comprising:
obtaining two-dimensional data and three-dimensional data associated with an environment of an autonomous vehicle at a current time;
processing the two-dimensional data with a machine-learned segmentation model to generate a two-dimensional segmentation for the environment;
generating a current three-dimensional segmentation based at least in part on the three-dimensional data and the two-dimensional segmentation;
identifying one or more historical three-dimensional datapoints previously generated for the environment at one or more times preceding the current time; and
determining one or more travel way features within the environment at the current time based on the current three-dimensional segmentation and the one or more historical three-dimensional datapoints previously generated for the environment at the one or more times preceding the current time.
12. The autonomous vehicle control system of claim 11 , wherein the operations further comprise:
initiating a motion of the autonomous vehicle based on the one or more travel way features within the environment.
13. The autonomous vehicle control system of claim 11 , wherein the operations further comprise:
updating a map based on the one or more travel way features within the environment.
14. The autonomous vehicle control system of claim 13 , wherein updating the map based on the one or more travel way features within the environment comprises:
determining a current position for the autonomous vehicle; and
updating the map based on the one or more travel way features within the environment and the current position of the autonomous vehicle.
15. The autonomous vehicle control system of claim 11 , wherein identifying the one or more historical three-dimensional datapoints previously generated for the environment at the one or more times preceding the current time comprises:
identifying the one or more historical three-dimensional datapoints previously generated for the environment at the one or more times preceding the current time based on a spatial relationship between the one or more historical three-dimensional datapoints and the current three-dimensional segmentation.
16. The autonomous vehicle control system of claim 11 , wherein the operations further comprise:
storing data indicative of the current three-dimensional segmentation in a classification database comprising the one or more historical three-dimensional datapoints previously generated for the environment at the one or more times preceding the current time.
17. An autonomous vehicle comprising:
one or more processors;
one or more tangible, non-transitory, computer readable media that collectively store instructions that when executed by the one or more processors cause the one or more processors to perform operations comprising:
obtaining two-dimensional data and three-dimensional data associated with an environment of the autonomous vehicle at a current time;
processing the two-dimensional data with a machine-learned segmentation model to generate a two-dimensional segmentation for the environment;
generating a current three-dimensional segmentation based at least in part on the three-dimensional data and the two-dimensional segmentation;
identifying one or more historical three-dimensional datapoints previously generated for the environment at one or more times preceding the current time; and
determining one or more travel way features within the environment at the current time based on the current three-dimensional segmentation and the one or more historical three-dimensional datapoints previously generated for the environment at the one or more times preceding the current time.
18. The autonomous vehicle of claim 17 , wherein the operations further comprise:
initiating a motion of the autonomous vehicle based on the one or more travel way features within the environment.
19. The autonomous vehicle of claim 17 , wherein the operations further comprise:
updating a map based on the one or more travel way features within the environment.
20. The autonomous vehicle of claim 19 , wherein updating the map based on the one or more travel way features within the environment comprises:
determining a current position for the autonomous vehicle; and
updating the map based on the one or more travel way features within the environment and the current position of the autonomous vehicle.