Jointly learnable behavior and trajectory planning for autonomous vehicles
Systems and methods for generating motion plans for autonomous vehicles are provided. An autonomous vehicle can include a machine-learned motion planning system including one or more machine-learned models configured to generate target trajectories for the autonomous vehicle. The model(s) include a behavioral planning stage configured to receive situational data based at least in part on the one or more outputs of the set of sensors and to generate behavioral planning data based at least in part on the situational data and a unified cost function. The model(s) includes a trajectory planning stage configured to receive the behavioral planning data from the behavioral planning stage and to generate target trajectory data for the autonomous vehicle based at least in part on the behavioral planning data and the unified cost function.
1 . An autonomous vehicle, comprising:
one or more processors; and
one or more non-transitory computer-readable media that store instructions that are executable by the one or more processors to cause the one or more processors to perform operations, the operations comprising:
obtaining, from a plurality of sensors of the autonomous vehicle, situational data associated with an environment external to the autonomous vehicle;
generating, by a machine-learned motion planning system and using a unified cost function, a plurality of possible trajectories based at least in part on the situational data; and
controlling a motion of the autonomous vehicle in the environment according to a trajectory of the plurality of possible trajectories selected, by the machine-learned motion planning system and using the unified cost function.
2 . The autonomous vehicle of claim 1 , wherein the plurality of possible trajectories are generated by sampling one or more values for the plurality of possible trajectories based at least in part on the situational data.
3 . The autonomous vehicle of claim 2 , wherein the one or more values comprise one or more lateral offset values determined with respect to a path.
4 . The autonomous vehicle of claim 1 , wherein a respective possible trajectory comprises a sampled longitudinal trajectory and a sampled lateral trajectory.
5 . The autonomous vehicle of claim 4 , wherein the sampled longitudinal trajectory is parameterized in terms of time, and wherein the sampled lateral trajectory is parameterized in terms of distance along the sampled longitudinal trajectory.
6 . The autonomous vehicle of claim 1 , wherein the unified cost function is optimized by training the machine-learned motion planning system end-to-end.
7 . The autonomous vehicle of claim 6 , wherein training machine-learned motion planning system end-to-end comprises, for a training input comprising training situational data and a ground truth output comprising a reference trajectory:
generating, by the machine-learned motion planning system using the unified cost function, a plurality of possible training trajectories based at least in part on the unified cost function and the training input;
selecting, by the machine-learned motion planning system using the unified cost function, a particular training trajectory of the plurality of possible training trajectories;
determining a loss based at least in part on the reference trajectory and the particular training trajectory; and
updating the unified cost function based at least in part on the loss.
8 . The autonomous vehicle of claim 7 , wherein training machine-learned motion planning system end-to-end comprises updating one or more parameters of the machine-learned motion planning system based at least in part on the loss.
9 . The autonomous vehicle of claim 1 , wherein the operations comprise:
determining, using the unified cost function, one or more control parameters for executing the particular trajectory.
10 . A computer-implemented method, comprising:
obtaining, from a plurality of sensors of an autonomous vehicle, situational data associated with an environment external to the autonomous vehicle;
generating, by a machine-learned motion planning system using a unified cost function, a plurality of possible trajectories based at least in part on the situational data; and
controlling a motion of the autonomous vehicle in the environment according to a trajectory of the plurality of possible trajectories selected by the machine-learned motion planning system and using the unified cost function.
11 . The computer-implemented method of claim 10 , wherein the plurality of possible trajectories are generated by sampling one or more values for the plurality of possible trajectories based at least in part on the situational data.
12 . The computer-implemented method of claim 11 , wherein the one or more values comprise one or more lateral offset values determined with respect to a path.
13 . The computer-implemented method of claim 10 , wherein a respective possible trajectory comprises a sampled longitudinal trajectory and a sampled lateral trajectory.
14 . The computer-implemented method of claim 13 , wherein the sampled longitudinal trajectory is parameterized in terms of time, and wherein the sampled lateral trajectory is parameterized in terms of distance along the sampled longitudinal trajectory.
15 . The computer-implemented method of claim 14 , wherein the unified cost function is optimized by training the machine-learned motion planning system end-to-end.
16 . The computer-implemented method of claim 15 , wherein training machine-learned motion planning system end-to-end comprises, for a training input comprising training situational data and a ground truth output comprising a reference trajectory:
generating, by the machine-learned motion planning system using the unified cost function, a plurality of possible training trajectories based at least in part on the unified cost function and the training input;
selecting, by the machine-learned motion planning system using the unified cost function, a particular training trajectory of the plurality of possible training trajectories;
determining a loss based at least in part on the reference trajectory and the particular training trajectory; and
updating the unified cost function based at least in part on the loss.
17 . The computer-implemented method of claim 16 , wherein training machine-learned motion planning system end-to-end comprises updating one or more parameters of the machine-learned motion planning system based at least in part on the loss.
18 . The computer-implemented method of claim 10 , comprising:
determining, using the unified cost function, one or more control parameters for executing the particular trajectory.
19 . One or more non-transitory computer-readable media that store instructions that are executable by one or more processors to cause a computing system to perform operations, the operations comprising:
obtaining, from a plurality of sensors of the autonomous vehicle, situational data associated with an environment external to the autonomous vehicle;
generating, by a machine-learned motion planning system and using a unified cost function, a plurality of possible trajectories based at least in part on the situational data; and
controlling a motion of the autonomous vehicle in the environment according to a trajectory of the plurality of possible trajectories selected, by the machine-learned motion planning system and using the unified cost function.
20 . The one or more non-transitory computer-readable media of claim 19 , wherein the plurality of possible trajectories are generated by sampling one or more values for the plurality of possible trajectories based at least in part on the situational data.