Systems and methods for generating behavioral predictions in reaction to autonomous vehicle movement
Systems and methods are directed to generating behavioral predictions in reaction to autonomous vehicle movement. In one example, a computer-implemented method includes obtaining, by a computing system, local scene data associated with an environment external to an autonomous vehicle, the local scene data including actor data for an actor in the environment external to the autonomous vehicle. The method includes extracting, by the computing system and from the local scene data, one or more actor prediction parameters for the actor using a machine-learned parameter extraction model. The method includes determining, by the computing system, a candidate motion plan for the autonomous vehicle. The method includes generating, by the computing system and using a machine-learned prediction model, a reactive prediction for the actor based at least in part on the one or more actor prediction parameters and the candidate motion plan.
1 . A computer-implemented method, comprising:
obtaining local scene data associated with an environment external to an autonomous vehicle, the local scene data comprising actor data for an actor in the environment external to the autonomous vehicle;
generating, from the local scene data and using a first machine-learned model, a latent space representation of at least a portion of the local scene data;
processing, based on inputting the latent space representation to a second machine-learned model, each trajectory of a plurality of candidate trajectories to generate a plurality of reactive predictions for the actor that are respectively conditioned on the plurality of candidate trajectories;
selecting a trajectory of the plurality of candidate trajectories using one or more cost functions and the plurality of reactive predictions; and
operating the autonomous vehicle based at least in part on the selected trajectory.
2 . The computer-implemented method of claim 1 , wherein:
the first machine-learned model generates the latent space representation once per evaluation cycle, the evaluation cycle including evaluating the plurality of candidate trajectories using the second machine-learned model.
3 . The computer-implemented method of claim 1 , wherein the second machine-learned model predicts actor behavior in response to multiple candidate trajectories using the latent space representation that is extracted once per evaluation cycle.
4 . The computer-implemented method of claim 1 , wherein the plurality of candidate trajectories is evaluated using a common set of feature extractions that includes the latent space representation.
5 . The computer-implemented method of claim 1 , wherein the second machine-learned model executes more quickly than the first machine-learned model.
6 . The computer-implemented method of claim 1 , wherein operating the autonomous vehicle based at least in part on the selected trajectory comprises:
for a respective candidate trajectory, assigning a candidate rating to the respective candidate trajectory based at least in part on a respective reactive prediction for the actor;
selecting, based at least in part on the candidate rating for the respective candidate trajectory, the selected trajectory; and
operating the autonomous vehicle based on the selected trajectory.
7 . The computer-implemented method of claim 1 , wherein generating the plurality of reactive predictions comprises:
inputting the latent space representation to the second machine-learned model;
for each respective candidate trajectory of the plurality of candidate trajectories:
inputting the respective candidate trajectory to the second machine-learned model;
generating, using the second machine-learned model, a respective reactive prediction for the actor based at least in part on the latent space representation and the respective candidate trajectory, the respective reactive prediction comprising a probability representing a likelihood of the actor reacting in a particular manner to a movement of the autonomous vehicle based on the respective candidate trajectory.
8 . The computer-implemented method of claim 1 , wherein the second machine-learned model outputs an occupancy map describing a location for the actor.
9 . The computer-implemented method of claim 1 , wherein the second machine-learned model outputs a yield probability for the actor.
10 . An autonomous vehicle control system for controlling an autonomous vehicle, the autonomous vehicle control system comprising:
one or more processors; and
one or more non-transitory computer-readable media storing instructions that are executable by the one or more processors to cause the autonomous vehicle control system to perform operations, the operations comprising:
obtaining local scene data associated with an environment external to the autonomous vehicle, the local scene data comprising actor data for an actor in the environment external to the autonomous vehicle;
generating, from the local scene data and using a first machine-learned model, a latent space representation of at least a portion of the local scene data;
processing, based on inputting the latent space representation to a second machine-learned model, each trajectory of a plurality of candidate trajectories to generate a plurality of reactive predictions for the actor that are respectively conditioned on the plurality of candidate trajectories;
selecting a trajectory of the plurality of candidate trajectories using one or more cost functions and the plurality of reactive predictions; and
operating the autonomous vehicle based at least in part on the selected trajectory.
11 . The autonomous vehicle control system of claim 10 , wherein:
the first machine-learned model generates the latent space representation once per evaluation cycle, the evaluation cycle including evaluating the plurality of candidate trajectories using the second machine-learned model.
12 . The autonomous vehicle control system of claim 10 , wherein the second machine-learned model predicts actor behavior in response to multiple candidate trajectories using the latent space representation that is extracted once per evaluation cycle.
13 . The autonomous vehicle control system of claim 10 , wherein the plurality of candidate trajectories is evaluated using a common set of feature extractions that includes the latent space representation.
14 . The autonomous vehicle control system of claim 10 , wherein the second machine-learned model executes more quickly than the first machine-learned model.
15 . The autonomous vehicle control system of claim 10 , wherein operating the autonomous vehicle based at least in part on the selected trajectory comprises:
for a respective candidate trajectory, assigning a candidate rating to the respective candidate trajectory based at least in part on a respective reactive prediction for the actor;
selecting, based at least in part on the candidate rating for the respective candidate trajectory, the selected trajectory; and
operating the autonomous vehicle based on the selected trajectory.
16 . The autonomous vehicle control system of claim 10 , wherein generating the plurality of reactive predictions comprises:
inputting the latent space representation to the second machine-learned model;
for each respective candidate trajectory of the plurality of candidate trajectories:
inputting the respective candidate trajectory to the second machine-learned model;
generating, using the second machine-learned model, a respective reactive prediction for the actor based at least in part on the latent space representation and the respective candidate trajectory, the respective reactive prediction comprising a probability representing a likelihood of the actor reacting in a particular manner to a movement of the autonomous vehicle based on the respective candidate trajectory.
17 . The autonomous vehicle control system of claim 10 , wherein the second machine-learned model outputs an occupancy map describing a location for the actor.
18 . The autonomous vehicle control system of claim 10 , wherein the second machine-learned model outputs a yield probability for the actor.
19 . One or more non-transitory computer-readable media storing instructions that are executable by one or more processors to cause an autonomous vehicle control system to perform operations, the operations comprising:
obtaining local scene data associated with an environment external to the autonomous vehicle, the local scene data comprising actor data for an actor in the environment external to the autonomous vehicle;
generating, from the local scene data and using a first machine-learned model, a latent space representation of at least a portion of the local scene data;
processing, based on inputting the latent space representation to a second machine-learned model, each trajectory of a plurality of candidate trajectories to generate a plurality of reactive predictions for the actor that are respectively conditioned on the plurality of candidate trajectories;
selecting a trajectory of the plurality of candidate trajectories using one or more cost functions and the plurality of reactive predictions; and
operating the autonomous vehicle based at least in part on the selected trajectory.
20 . The one or more non-transitory computer-readable media of claim 19 , wherein:
the first machine-learned model generates the latent space representation once per evaluation cycle, the evaluation cycle including evaluating the plurality of candidate trajectories using the second machine-learned model.