System for generating scene context data using a reference graph
Techniques for improving operational decisions of an autonomous vehicle are discussed herein. In some cases, a system may generate reference graphs associated with a route of the autonomous vehicle. Such reference graphs can comprise precomputed feature vectors based on grid regions and/or lane segments. The feature vectors are usable to determine scene context data associated with static objects to reduce computational expenses and compute time.
1. A system comprising:
one or more processors; and
one or more non-transitory computer readable media storing instructions executable by the one or more processors, wherein the instructions, when executed, cause the system to perform operations comprising:
receiving sensor data associated with a physical environment surrounding an autonomous vehicle;
receiving a reference graph associated with the physical environment, the reference graph comprising a plurality of nodes, individual nodes of the plurality of nodes comprising a respective feature vector representative of object data;
determining, based at least in part on the sensor data, a state of an object within the physical environment;
determining, based at least in part on the state of the object and the reference graph, a number of nodes of the plurality of nodes;
generating, based at least in part on feature vectors of the number of nodes, scene context data associated with the physical environment, wherein the feature vectors of the reference graph are computed prior to receiving the reference graph and prior to receiving the sensor data; and
controlling the autonomous vehicle based at least in part on the scene context data.
2. The system of claim 1 , the operations further comprising:
determining, based at least in part on the scene context data, a future state of the object relative to the physical environment;
determining, based at least in part on the future state of the object and the reference graph, a second number of nodes of the plurality of nodes;
generating, based at least in part on the second number of nodes of the reference graph, second scene context data associated with the physical environment; and
wherein controlling the autonomous vehicle is based at least in part on the second scene context data.
3. The system of claim 1 , wherein the individual nodes of the reference graph correspond to a respective discrete portion of the physical environment.
4. The system of claim 1 , wherein determining the number of nodes of the plurality of nodes is based at least in part on a position of the object.
5. The system of claim 1 , wherein determining the number of nodes of the plurality of nodes is based at least in part on a physical distance or graph distance between the individual nodes of the plurality of nodes and the object.
6. The system of claim 5 , wherein determining the number of nodes is based on a heuristic associated with characteristics of the physical environment.
7. One or more non-transitory computer-readable media storing instructions that, when executed, cause one or more processors to perform operations comprising:
determining, based on sensor data, state data associated with an object and a physical environment proximate to the object;
receiving, based at least in part on the state data and a reference graph, a subset of the reference graph comprising two or more nodes of the reference graph;
determining, based at least in part on the two or more nodes, feature vectors representing scene context data associated with the physical environment, wherein the feature vectors are computed prior to receiving the reference graph and prior to receiving the sensor data; and
controlling an autonomous vehicle based at least in part on the scene context data.
8. The one or more non-transitory computer-readable media of claim 7 , wherein receiving the reference graph further comprises:
selecting the two or more nodes from a plurality of nodes based at least in part on the state data and a characteristic of the physical environment.
9. The one or more non-transitory computer-readable media of claim 7 , wherein the state data comprises position data of the object and receiving the reference graph further comprises:
selecting the two or more nodes from a plurality of nodes based at least in part on the position data of the object.
10. The one or more non-transitory computer-readable media of claim 7 , wherein receiving the reference graph further comprises:
selecting the two or more nodes from a plurality of nodes is based at least in part on a heuristic.
11. The one or more non-transitory computer-readable media of claim 7 , the operations further comprising
interpreting the scene context data from context data of the two or more nodes.
12. The one or more non-transitory computer-readable media of claim 7 , wherein the receiving the subset of the reference graph is based at least in part on a route associated with the autonomous vehicle.
13. A method comprising:
determining, based on sensor data, state data associated with an object and a physical environment surrounding the object;
receiving, based at least in part on the state data and a reference graph, a subset of the reference graph comprising two or more nodes of the reference graph;
determining, based at least in part on the two or more nodes, feature vectors representing scene context data associated with the physical environment, wherein the feature vectors are computed prior to receiving the reference graph and prior to receiving the sensor data; and
controlling an autonomous vehicle based at least in part on the scene context data.
14. The method of claim 13 , wherein the state data is current state data and the method further comprises:
determining, based at least in part on the current state data and the scene context data, future state data of the object relative to the physical environment;
determining, based at least in part on the future state data of the object and the reference graph, a second subset of the reference graph;
generating, based at least in part on the second subset of the reference graph, second scene context data associated with the physical environment; and
wherein controlling the autonomous vehicle is based at least in part on the second scene context data.
15. The method of claim 13 , wherein the state data comprises position data of the object and receiving the reference graph further comprises:
selecting the two or more nodes from a plurality of nodes based at least in part on the position data of the object.
16. The method of claim 13 , wherein receiving the reference graph further comprises:
selecting the two or more nodes from a plurality of nodes based at least in part on the state data and a characteristic of the physical environment.
17. The method of claim 13 , wherein controlling the autonomous vehicle further comprises determining a trajectory associated with the autonomous vehicle based at least in part on the scene context data.
18. The method of claim 13 , wherein controlling the autonomous vehicle further comprises determining a pre-planned route associated with the autonomous vehicle based at least in part on the scene context data.
19. The system of claim 1 , the operations further comprising:
determining, based at least in part on a future state of the object and the reference graph, a second number of nodes of the plurality of nodes; and
generating, based at least in part on the second number of nodes of the reference graph, second scene context data associated with the physical environment.
20. The one or more non-transitory computer-readable media of claim 7 , the operations further comprising:
determining, based at least in part on a future state of the object and the reference graph, a second subset of nodes of the two or more nodes; and
generating, based at least in part on the second subset of nodes, second scene context data associated with the physical environment.