Systems and Methods for Streaming Processing for Autonomous Vehicles
Generally, the present disclosure is directed to systems and methods for streaming processing within one or more systems of an autonomy computing system. When an update for a particular object or region of interest is received by a given system, the system can control transmission of data associated with the update as well as a determination of other aspects by the given system. For example, the system can determine based on a received update for a particular aspect and a priority classification and/or interaction classification determined for that aspect whether data associated with the update should be transmitted to a subsequent system before waiting for other updates to arrive.
1 . An autonomy computing system, comprising:
one or more processors; and
one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the one or more processors to perform operations, the operations comprising:
obtaining by a first system of an autonomy computing system a first portion of autonomy data descriptive of an update for a first feature of a first object detected within a surrounding environment of an autonomous vehicle;
determining that a second object has an interaction dependence relative to the first object;
determining an expected time estimate for receipt by the first system of a second portion of autonomy data descriptive of an update for a first feature of the second object;
determining a second feature of the first object based at least in part on the first feature of the first object; and
coordinating transmission of the second feature of the first object to a second system of the autonomy computing system based at least in part on the expected time estimate.
2 . The autonomy computing system of claim 1 , wherein determining the expected time estimate comprises determining that the expected time estimate is less than a threshold value, and wherein coordinating transmission of the second feature of the first object to the second system of the autonomy computing system based at least in part on the expected time estimate comprises:
in response to determining that the expected time estimate is less than the threshold value, obtaining by the first system the second portion of autonomy data descriptive of an update for the first feature of the second object;
determining a second feature of the second object based at least in part on the first feature of the second object; and
transmitting the second feature of the first object and the second feature of the second object from the first system to the second system.
3 . The autonomy computing system of claim 1 , wherein determining the expected time estimate comprises determining that the expected time estimate is greater than a threshold value, and wherein coordinating transmission of the second feature of the first object to the second system of the autonomy computing system based at least in part on the expected time estimate comprises:
in response to determining that the expected time estimate is greater than the threshold value, determining, by the first system, a predicted second feature of the second object based at least in part on the first feature of the first object; and
providing the second feature of the first object and the predicted second feature of the second object from the first system to the second system.
4 . The autonomy computing system of claim 1 , wherein determining that the second object has an interaction dependence relative to the first object comprises determining that the first object and the second object are likely to interact given their current trajectories.
5 . The autonomy computing system of claim 1 , wherein determining that the second object has an interaction dependence relative to the first object comprises determining that the first object and the second object are within a predetermined level of proximity to one another at one or more of a current time or a future time.
6 . The autonomy computing system of claim 1 , wherein:
the first system comprises a perception system configured to generate state data descriptive of at least a current state of each of a plurality of objects that are perceived by the autonomous vehicle;
the first feature comprises sensor data associated with an object; and
the second feature comprises state data for the object determined at least in part from the sensor data associated with the object.
7 . The autonomy computing system of claim 6 , wherein the second system comprises a prediction system configured to determine, for a current time frame, a predicted track for each of the plurality of objects including the first object and the second object.
8 . The autonomy computing system of claim 1 , wherein:
the first system comprises a prediction system configured to determine a predicted track for each of a plurality of objects that are perceived by the autonomous vehicle;
the first feature comprises current state data associated with an object of the plurality of objects; and
the second feature comprises predicted future track data for the object determined at least in part from the current state data associated with the object.
9 . The autonomy computing system of claim 8 , wherein the second system comprises a motion planning system configured to determine, for a current time frame, a motion plan for the autonomous vehicle based at least in part on track data for the first object and the second object.
10 . The autonomy computing system of claim 1 , further comprising determining a priority classification associated with the first object; and wherein coordinating transmission of the second feature of the first object to a second system is further based at least in part on the priority classification.
11 . The autonomy computing system of claim 10 , wherein determining a priority classification for each object in the plurality of objects comprises classifying each object as either high-priority or low-priority.
12 . The autonomy computing system of claim 1 , wherein the operations further comprise:
obtaining, by the first system, autonomy data descriptive of updates for a first feature of a plurality of objects detected within a surrounding environment of an autonomous vehicle;
determining a priority classification for each object in the plurality of objects based at least in part on the respective updates for the first feature of each object;
determining an order in which the one or more processors determines a second feature for each object based at least in part on the priority classification for each object; and
determining the second feature for each object based at least in part on the determined order.
13 . The autonomy computing system of claim 1 , wherein determining that the second object has an interaction dependence relative to the first object comprises accessing an object dependence graph that identifies interacting objects.
14 . The autonomy computing system of claim 1 , wherein determining that the second object has an interaction dependence relative to the first object comprises determining an interaction classification for the first object using a machine-learned model that has been trained based at least in part on training data that comprises annotated vehicle data logs that were previously collected during previous autonomous vehicle driving sessions.
15 . A computer-implemented method, comprising:
obtaining, by a computing system comprising one or more computing devices, a first portion of autonomy data descriptive of an update for a first feature of a first object detected within a surrounding environment of an autonomous vehicle;
determining, by the computing system, that a second object has an interaction dependence relative to the first object;
determining, by the computing system, an expected time estimate for receiving a second portion of autonomy data descriptive of an update for a first feature of the second object;
determining, by the computing system, a second feature of the first object based at least in part on the first feature of the first object; and
coordinating, by the computing system, transmission of the second feature of the first object to a second system based at least in part on the expected time estimate.
16 . The computer-implemented method of claim 15 , wherein determining the expected time estimate comprises determining that the expected time estimate is less than a threshold value, and wherein coordinating transmission of the second feature of the first object to the second system based at least in part on the expected time estimate comprises:
in response to determining that the expected time estimate is less than the threshold value, obtaining, by the first system, the second portion of autonomy data descriptive of an update for the first feature of the second object;
determining a second feature of the second object based at least in part on the first feature of the second object; and
transmitting the second feature of the first object and the second feature of the second object from the first system to the second system.
17 . The computer-implemented method of claim 15 , wherein determining the expected time estimate comprises determining that the expected time estimate is greater than a threshold value, and wherein coordinating transmission of the second feature of the first object to the second system based at least in part on the expected time estimate comprises:
in response to determining that the expected time estimate is greater than the threshold value, determining, by the first system, a predicted second feature of the second object based at least in part on the first feature of the first object; and
providing the second feature of the first object and the predicted second feature of the second object from the first system to the second system.
18 . An autonomous vehicle, comprising:
one or more processors; and
one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the one or more processors to perform operations, the operations comprising:
obtaining by a first system of an autonomous vehicle a first portion of autonomy data descriptive of an update for a first feature of a first object detected within a surrounding environment of an autonomous vehicle;
determining that a second object has an interaction classification indicative of an interaction dependence relative to the first object, the interaction classification being indicative of one or more predetermined types of interaction between objects;
determining an expected time estimate for receiving a second portion of autonomy data descriptive of an update for a first feature of the second object;
determining a second feature of the first object based at least in part on the first feature of the first object; and
coordinating transmission of the second feature of the first object to the second system based at least in part on the expected time estimate.
19 . The autonomous vehicle of claim 18 , wherein:
the first system comprises a perception system configured to generate state data descriptive of at least a current state of each of a plurality of objects that are perceived by the autonomous vehicle;
the second system comprises a prediction system configured to determine, for a current time frame, a predicted track for each of the plurality of objects including the first object and the second object;
the first feature comprises sensor data; and
the second feature comprises state data determined at least in part from the sensor data associated.
20 . The autonomous vehicle of claim 18 , wherein:
the first system comprises a prediction system configured to determine a predicted track for each of a plurality of objects that are perceived by the autonomous vehicle;
the second system comprises a motion planning system configured to determine, for a current time frame, a motion plan for the autonomous vehicle based at least in part on track data for the first object and the second object;
the first feature comprises current state data; and
the second feature comprises predicted future track data determined at least in part from the current state data.