Systems and methods for segregating machine learned models for distributed processing
Systems and methods of the present disclosure are directed to a method for editing a machine-learned model to facilitate distributed processing. The method can include obtaining a machine-learned graph model comprising a plurality of connected nodes. The method can include determining a plurality of processing capabilities for a respective plurality of computation resources. The method can include determining a plurality of portions from the machine-learned graph model, wherein each of the plurality of portions comprises a respective subset of the plurality of nodes and a minimum processing capability. The method can include assigning each of the plurality of portions to a respective computation resource of the plurality of computation resources based at least in part on the minimum processing capability of a respective portion and the processing capability of the respective computation resource.
1 . A computer-implemented method for editing a machine-learned model to facilitate distributed processing, the method comprising:
obtaining, by a computing system comprising one or more computing devices, a machine-learned graph model comprising a plurality of connected nodes;
determining, by the computing system, a plurality of processing capabilities for a respective plurality of computation resources of a vehicle computing system onboard an autonomous vehicle;
determining, by the computing system, a plurality of portions from the machine-learned graph model, wherein each of the plurality of portions comprises a respective subset of the plurality of nodes and a minimum processing requirement;
assigning, by the computing system, each of the plurality of portions to a respective computation resource of the plurality of computation resources onboard the autonomous vehicle based at least in part on the minimum processing requirement of a respective portion and the processing capability of the respective computation resource; and
providing, by the computing system, each of the plurality of portions to a respective computation resource to which the portion is assigned, wherein the plurality of computation resources are configured to process the plurality of portions.
2 . The computer-implemented method of claim 1 , wherein the method further comprises:
generating, by the computing system for each of the plurality of portions, one or more connections between the portion and one or more other portions to obtain a reconstructed machine-learned graph model; and
providing, by the computing system, the reconstructed machine-learned graph model to the plurality of computation resources, wherein the plurality of computation resources is configured to process the reconstructed machine-learned graph model.
3 . The computer-implemented method of claim 1 , wherein the method further comprises:
obtaining, by the computing system, data descriptive of a processing performance of each of the plurality of computation resources; and
applying, by the computing system, one or more optimizations to one or more portions of the plurality of portions to obtain an optimized machine-learned graph model, wherein the one or more optimizations are based at least in part on the processing performance of one or more computation resources assigned to the one or more portions.
4 . The computer-implemented method of claim 3 , wherein the method further comprises:
providing, by the computing system, the optimized machine-learned graph model to the plurality of computation resources for processing; and
obtaining, by the computing system, data indicative of an updated processing performance of at least one of the plurality of computation resources, the updated processing performance of the at least one computation resource being different than the processing performance of the at least one computation resource.
5 . The computer-implemented method of claim 3 , wherein the one or more optimizations comprise at least one of:
adjusting, by the computing system, the minimum processing requirement of each of the one or more portions; or
compressing, by the computing system, the subset of nodes of each of the one or more portions.
6 . The computer-implemented method of claim 3 , wherein:
for each of the plurality of portions, the one or more optimizations are further based at least in part on an autonomous vehicle processing task respectively associated with the portion; and
the autonomous vehicle processing task comprises at least one of:
a sensor data preprocessing task, wherein sensor data comprises image data, LIDAR data, RADAR data, heat sensor data, or ultrasonic sensor data;
a sensor data processing task;
a segmentation task
a detection task
an estimation task; or
one or more mathematical operations.
7 . The computer-implemented method of claim 1 , wherein:
each of the plurality of nodes is connected to at least one other node of the plurality of nodes with at least one respective node connection; and
each of the plurality of nodes comprises one or more neural units of a neural network.
8 . The computer-implemented method of claim 1 , wherein:
the subset of nodes of a first portion of the plurality of portions comprises a first node and a second node;
the minimum processing requirement of the first node is higher than the minimum processing requirement of the second node; and
the minimum processing requirement of the first portion is equivalent to the minimum processing requirement of the first node.
9 . The computer-implemented method of claim 1 , wherein each of the plurality of computation resources comprises at least a portion of at least one of:
an application-specific integrated circuit;
a memory;
a processor;
a processor core;
a field-programmable gate array;
an accelerator;
a tensor processing unit; or
a graphics processing unit.
10 . The computer-implemented method of claim 1 , wherein each of the plurality of processing capabilities and the plurality of processing capabilities comprises at least one of:
a floating point precision;
a processing throughput;
an output accuracy; or
a processing latency.
11 . A computing system of an autonomous vehicle, comprising:
a plurality of computation resources comprising a respective plurality of processing capabilities;
one or more tangible, non-transitory computer readable media storing computer-readable instructions that when executed by at least one of the plurality of computation resources cause the at least one of the plurality of computation resources to perform operations, the operations comprising:
obtaining an optimized machine-learned graph model comprising a plurality of portions and portion assignment data descriptive of an assignment of each of the plurality of portions to a respective computation resource of the plurality of computation resources, wherein the plurality of portions are associated with a respective plurality of minimum processing capabilities, wherein the portion assignment data is based at least in part on the plurality of processing capabilities and the plurality of minimum processing capabilities;
obtaining input data from one or more systems of the autonomous vehicle, wherein the input data is associated with an autonomous vehicle processing task;
processing, using the respective computation resources of the plurality of computation resources of the computing system of the autonomous vehicle, the input data with the optimized machine-learned graph model based at least in part on the portion assignment data to obtain output data associated with the autonomous vehicle processing task; and
providing the output data to one or more additional systems of the autonomous vehicle.
12 . The computing system of claim 11 , wherein processing the input data with the optimized machine-learned graph model based at least in part on the portion assignment data comprises processing each of the plurality of portions with the computation resource to which a respective portion is assigned by the portion assignment data to obtain the output data associated with the autonomous vehicle processing task.
13 . The computing system of claim 11 , wherein the operations further comprise:
providing data descriptive of a processing performance of each of the plurality of computation resources to a computing system associated with a service entity; and
in response to providing the data descriptive of the processing performance, obtaining at least one of an updated machine-learned graph model different than the optimized machine-learned graph model or updated portion assignment data different than the portion assignment data.
14 . The computing system of claim 11 , wherein the autonomous vehicle task comprises at least one of:
a perception task;
a prediction task; or
a motion planning task.
15 . The computing system of claim 11 , wherein the autonomous vehicle processing task is associated with a service of the service entity.
16 . The computing system of claim 11 , wherein each of the plurality of computation resources comprises at least a portion of at least one of:
an application-specific integrated circuit;
a memory;
a processor;
a processor core;
a field-programmable gate array;
an accelerator;
a tensor processing unit; or
a graphics processing unit.
17 . The computing system of claim 11 , wherein a processing capability and a minimum processing requirement comprises at least one of:
a floating point precision;
a processing throughput;
an output accuracy; or
a processing latency.
18 . One or more tangible, non-transitory computer readable media storing computer-readable instructions that when executed by one or more processors cause the one or more processors to perform operations, the operations comprising:
obtaining a graph model comprising a plurality of connected nodes;
determining a plurality of processing capabilities for a respective plurality of computation resources of a vehicle computing system onboard an autonomous vehicle;
determining a plurality of portions from the graph model, wherein each of the plurality of portions comprises a respective subset of the plurality of nodes and a minimum processing requirement;
assigning each of the plurality of portions to a respective computation resource of the plurality of computation resources of the vehicle computing system onboard the autonomous vehicle based at least in part on the minimum processing requirement of a respective portion and the processing capability of the respective computation resource; and
providing each of the plurality of portions to a respective computation resource to which the portion is assigned, wherein the respective computation resource is configured to process the respective portion.
19 . The one or more tangible, non-transitory computer readable media of claim 18 , wherein the operations further comprise:
generating, for each of the plurality of portions, one or more connections between the portion and one or more other portions to obtain a reconstructed graph model;
providing the reconstructed graph model to the plurality of computation resources, wherein the plurality of computation resources is configured to process the reconstructed graph model;
obtaining data descriptive of a processing performance of each of the plurality of computation resources;
applying one or more optimizations to at least one portion of the plurality of portions to obtain an optimized graph model, wherein the one or more optimizations are based at least in part on the processing performance of one or more computation resources assigned to the at least one portions;
providing the optimized graph model to the plurality of computation resources for processing; and
obtaining data indicative of an updated processing performance of each of the plurality of computation resources, the updated processing performance being different than the processing performance.