Systems and methods for generating motion forecast data for actors with respect to an autonomous vehicle and training a machine learned model for the same
Systems and methods for generating motion forecast data for actors with respect to an autonomous vehicle and training a machine learned model for the same are disclosed. The computing system can include an object detection model and a graph neural network including a plurality of nodes and a plurality of edges. The computing system can be configured to input sensor data into the object detection model; receive object detection data describing the location of the plurality of the actors relative to the autonomous vehicle as an output of the object detection model; input the object detection data into the graph neural network; iteratively update a plurality of node states respectively associated with the plurality of nodes; and receive, as an output of the graph neural network, the motion forecast data with respect to the plurality of actors.
1. A computer-implemented method comprising:
receiving, from one or more sensors, sensor data associated with an environment of an autonomous vehicle;
generating, using an object detection model, object detection data based on the sensor data, the object detection data describing a location of an actor relative to the autonomous vehicle;
generating, using a graph neural network, motion forecast data associated with the actor, wherein the graph neural network comprises a plurality of nodes and a plurality of edges, the plurality of nodes indicative of a plurality of actors including the actor and the edges indicative of relationships among the plurality of actors,
wherein the graph neural network is configured to update at least one node state of a first node by passing at least one message between at least two nodes comprising the first node; and
determining a motion plan for the autonomous vehicle based on the motion forecast data; and
controlling the autonomous vehicle based on the motion plan.
2. The computer-implemented method claim 1 , comprising:
obtaining map data associated with the environment of the autonomous vehicle;
concatenating, using the object detection model, the map data and the sensor data; and
generating, using the object detection model, the object detection data based on the map data and the sensor data.
3. The computer-implemented method of claim 1 , wherein the graph neural network is indicative of at least one of: (i) a partially connected graph neural network, or (ii) a fully connected graph neural network.
4. The computer-implemented method of claim 1 , wherein the at least one node state is indicative of at least one of: (i) a future trajectory of a respective actor of the plurality of actors or (ii) a behavior of the respective actor.
5. The computer-implemented method of claim 1 , wherein passing the at least one message between the at least two nodes is indicative of a potential interaction among the actors respectively associated with the at least two nodes.
6. The computer-implemented method of claim 5 , wherein the at least one message is indicative of a spatial relationship.
7. The computer-implemented method of claim 1 , wherein the sensor data comprises three-dimensional LiDAR data.
8. A vehicle computing system comprising:
one or more processors; and
one or more memory resources storing instructions executable by the one or more processors, to cause the one or more processors to:
receive, from one or more sensors, sensor data associated with an environment of an autonomous vehicle;
generate, using an object detection model, object detection data based on the sensor data, the object detection data describing a location of an actor relative to the autonomous vehicle;
generate, using a graph neural network, motion forecast data associated with the actor, wherein the graph neural network comprises a plurality of nodes and a plurality of edges, the plurality of nodes indicative of a plurality of actors including the actor and the edges indicative of relationships among the plurality of actors,
wherein the graph neural network is configured to update at least one node state of a first node by passing at least one message between at least two nodes comprising the first node; and
determine a motion plan for the autonomous vehicle based on the motion forecast data; and
control the autonomous vehicle based on the motion plan.
9. The vehicle computing system of claim 8 , wherein the instructions cause the one or more processors to:
obtain map data associated with the environment of the autonomous vehicle;
concatenate, using the object detection model, the map data and the sensor data; and
generate, using the object detection model, the object detection data based on the map data and the sensor data.
10. The vehicle computing system of claim 8 , wherein t graph neural network is indicative of at least one of: (i) a partially connected graph neural network, or (ii) a fully connected graph neural network.
11. The vehicle computing system of claim 8 , wherein the at least one node state is indicative of at least one of: (i) a future trajectory of a respective actor of the plurality of actors or (ii) a behavior of the respective actor.
12. The vehicle computing system of claim 8 , wherein passing the at least one message between the at least two nodes is indicative of a potential interaction among the actors respectively associated with the at least two nodes.
13. The vehicle computing system of claim 12 , wherein the at least one message is indicative of a spatial relationship.
14. The vehicle computing system of claim 8 , wherein the sensor data comprises three-dimensional LiDAR data.
15. A non-transitory computer-readable medium storing instructions executable by one or more processors of an autonomous vehicle computing system, to cause the one or more processors to:
receive, from one or more sensors, sensor data associated with an environment of an autonomous vehicle;
generate, using an object detection model, object detection data based on the sensor data, the object detection data describing a location of an actor relative to the autonomous vehicle;
generate, using a graph neural network, motion forecast data associated with the actor, wherein the graph neural network comprises a plurality of nodes and a plurality of edges, the plurality of nodes indicative of a plurality of actors including the actor and the edges indicative of relationships among the plurality of actors,
wherein the graph neural network is configured to update at least one node state of a first node by passing at least one message between at least two nodes comprising the first node; and
determine a motion plan for the autonomous vehicle based on the motion forecast data; and
control the autonomous vehicle based on the motion plan.
16. The non-transitory computer-readable medium of claim 15 , wherein the instructions cause the one or more processors to:
obtain map data associated with the environment of the autonomous vehicle;
concatenate, using the object detection model, the map data and the sensor data; and
generate, using the object detection model, the object detection data based on the map data and the sensor data.
17. The non-transitory computer-readable medium of claim 15 , wherein the graph neural network is indicative of at least one of (i) a partially connected graph neural network, or (ii) a fully connected graph neural network.
18. The non-transitory computer-readable medium of claim 15 , wherein the at least one node state is indicative of at least one of: (i) a future trajectory of a respective actor of the plurality of actors or (ii) a behavior of the respective actor.
19. The non-transitory computer-readable medium of claim 15 , wherein passing the at least one message between the at least two nodes is indicative of a potential interaction among the actors respectively associated with the at least two nodes.
20. The non-transitory computer-readable medium of claim 19 , wherein the at least one message is indicative of a spatial relationship.