Out-of-order workload execution
A computing system can include a sensor data input chiplet to (i) obtain sensor data from a sensor system, and (ii) store the sensor data in a cache. One or more workload processing chiplets can perform out-of-order execution on workloads corresponding to the sensor data stored in the cache, where respective workloads comprise sets of dependencies. The one or more workload processing chiplets perform out-of-order execution on the workloads by dynamically determining whether one or more dependencies exists for the respective workload, and when no data dependencies exist for the respective workload, executing the respective workload.
1 . A computing system for operating a vehicle, the computing system comprising:
a sensor data input chiplet to continuously (i) obtain sensor data from a sensor system of the vehicle when the vehicle is being operated, (ii) store the sensor data in a cache; and
a reservation table that includes a reference for each workload of a plurality of workloads, the reference for each workload identifying corresponding sensor data in the cache and dependency information for each workload;
one or more workload processing chiplets to execute a plurality of workloads, corresponding to driving tasks of the vehicle, using corresponding sensor data stored in the cache, the one or more workload processing chiplets to perform out-of-order execution on at least some of the plurality of workloads, by:
continuously monitoring the reservation table to determine available workloads that are ready for execution, each available workload referencing dependency information that indicates the workload has no unresolved dependency;
upon determining each available workload, executing the workload using the corresponding sensor data that is referenced for the workload; and
for workloads that, based on their dependency information referenced by the reservation table, are determined to not be ready for execution, buffering each of the respective workloads until the workload is determined to be ready for execution, then executing the respective workload using the corresponding sensor data that is referenced for the workload.
2 . The computing system of claim 1 , wherein the sensor data input chiplet further generates a plurality of workload entries for the plurality of workloads, the plurality of workload entries being included in the reservation table, each workload entry referencing corresponding sensor data in the cache and dependency information for the respective workload.
3 . The computing system of claim 2 , wherein upon executing individual workloads of the plurality of workloads, the one or more workload processing chiplets update, in the reservation table, dependency information referenced by workload entries of other workloads that have a dependency on the executed workload.
4 . The computing system of claim 2 , wherein the sensor data input chiplet generates a workload entry for each item of the sensor data stored in the cache, and references the workload entry for each item of the sensor data in the reservation table.
5 . The computing system of claim 2 , wherein the one or more workload processing chiplets monitor the reservation table to dynamically determine whether the dependency information referenced with each workload entry identifies any unresolved dependencies.
6 . The computing system of claim 5 , wherein the one or more workload processing chiplets monitor a workload window of the reservation table to determine a sequence of workload entries for available workloads using an instruction pointer, and for each workload that is determined to not be ready for execution, the one or more workload processing chiplets identify a next workload entry of the sequence to determine whether the next workload entry is ready for execution.
7 . The computing system of claim 1 , wherein the computing system is included on an autonomous vehicle, and wherein the sensor system comprises a sensor system of the autonomous vehicle.
8 . The computing system of claim 7 , wherein the sensor system comprises any combination of LIDAR sensors, image sensors, radar sensors, and ultrasonic sensors.
9 . The computing system of claim 7 , wherein the corresponding driving tasks of the vehicle include a plurality of the following: image stitching tasks, sensor fusion tasks, machine learning inference tasks, object detection tasks, object classification tasks, scene understanding tasks, right-of-way determination tasks, motion planning tasks, or autonomous vehicle control tasks.
10 . A non-transitory computer readable medium storing instructions that, when executed by one or more processors of a computing system for operating a vehicle, cause the computing system to:
on a sensor data input chiplet of the computing system, continuously (i) obtain sensor data from a sensor system, and (ii) store the sensor data in a cache;
implement a reservation table that includes a reference for each workload of a plurality of workloads, the reference for each workload identifying corresponding sensor data in the cache and dependency information for each workload;
on one or more workload processing chiplets of the computing system, execute a plurality of workloads, corresponding to driving tasks of the vehicle, using corresponding sensor data stored in the cache, the one or more workload processing chiplets to perform out-of-order execution on at least some of the plurality of workloads, by:
continuously monitoring the reservation table to determine available workloads that are ready for execution, each available workload referencing dependency information that indicates the workload has no unresolved dependency;
upon determining each available workload, executing the workload using the corresponding sensor data that is referenced for the workload; and
for workloads that, based on their dependency information referenced by the reservation table, are determined to not be ready for execution, buffering each of the respective workloads until the workload is determined to be ready for execution, then executing the respective workload using the corresponding sensor data that is referenced for the workload.
11 . The non-transitory computer readable medium of claim 10 , wherein the sensor data input chiplet further generates a plurality of workload entries for the plurality of workloads, the plurality of workload entries being included in the reservation table, each workload entry referencing corresponding sensor data in the cache and dependency information for the respective workload.
12 . The non-transitory computer readable medium of claim 11 , wherein upon executing individual workloads of the plurality of workloads, the one or more workload processing chiplets update, in the reservation table, dependency information referenced by workload entries of other workloads that have a dependency on the executed workload.
13 . The non-transitory computer readable medium of claim 11 , wherein the sensor data input chiplet generates a workload entry for each item of the sensor data stored in the cache, and references the workload entry for each item of the sensor data in the reservation table.
14 . The non-transitory computer readable medium of claim 11 , wherein the one or more workload processing chiplets monitor the reservation table to dynamically determine whether the dependency information referenced with each workload entry identifies any unresolved dependencies.
15 . The non-transitory computer readable medium of claim 14 , wherein the one or more workload processing chiplets monitor a workload window of the reservation table for to determine a sequence of workload entries for available workloads using an instruction pointer, and for each workload that is determined to not be ready for execution, the one or more workload processing chiplets identify a next workload entry of the sequence to determine whether the next workload entry is ready for execution.
16 . The non-transitory computer readable medium of claim 10 , wherein the computing system is included on an autonomous vehicle, and wherein the sensor system comprises a sensor system of the autonomous vehicle.
17 . The non-transitory computer readable medium of claim 16 , wherein the sensor system comprises any combination of LIDAR sensors, image sensors, radar sensors, and ultrasonic sensors.
18 . The non-transitory computer readable medium of claim 16 , wherein the corresponding driving tasks of the vehicle include a plurality of the following: image stitching tasks, sensor fusion tasks, machine learning inference tasks, object detection tasks, object classification tasks, scene understanding tasks, right-of-way determination tasks, motion planning tasks, or autonomous vehicle control tasks.
19 . A computer-implemented method for operating a vehicle, the method being performed by one or more processors of a computing system and comprising:
continuously (i) obtaining sensor data from a sensor system, and (ii) storing the sensor data in a cache;
executing a plurality of workloads, corresponding to driving tasks of the vehicle, using corresponding sensor data stored in the cache, including performing out-of-order execution on at least some of the plurality of workloads, by:
continuously monitoring a reservation table to determine available workloads that are ready for execution, each available workload referencing, in the reservation table, dependency information that indicates the workload has no unresolved dependency;
upon determining each available workload, executing the workload using the corresponding sensor data that is referenced in the reservation table for the workload; and
wherein for workloads that, based on their dependency information referenced by the reservation table, are determined to not be ready for execution, performing out-of-order execution on at least some of the plurality of workloads further comprises buffering each of the respective workloads until the workload is determined to be ready for execution, then executing the respective workload using the corresponding sensor data that is referenced for the workload in the reservation table.
20 . The method of claim 19 , wherein the method further comprises generating a plurality of workload entries for the plurality of workloads, the plurality of workload entries being included in the reservation table, each workload entry referencing corresponding sensor data in the cache and dependency information for the respective workload.