Allocating resources for a machine learning model
A method for allocating resources for a machine learning model is disclosed. A machine learning model to be executed on a special purpose machine learning model processor is received. A computational data graph is generated from the machine learning model. The computational dataflow graph represents the machine learning model which includes nodes, connector directed edges, and parameter directed edges. The operations of the computational dataflow graph is scheduled and then compiled using a deterministic instruction set architecture that specifies functionality of a special purpose machine learning model processor. An amount of resources required to execute the computational dataflow graph is determined. Resources are allocated based on the determined amounts of resources required to execute the machine learning model represented by the computational dataflow graph.
1. A computer-implemented method for allocating resources for a machine learning model, the method comprising:
receiving a machine learning model to execute on a special purpose machine learning model processor;
generating, from the machine learning model, a computational dataflow graph representing the machine learning model comprising a plurality of nodes, a plurality of connector directed edges, and a plurality of parameter directed edges, wherein each node represents a respective operation performed by the machine learning model as part of the model output from the model input, wherein each connector directed edge connects a respective first node to a respective second node that represents an operation that receives, as input, an output of an operation represented by the respective first node, and wherein each parameter directed edge connects into a respective node and represents a flow of one or more parameters of the model as input to the operation represented by the respective node;
scheduling operations of the computational dataflow graph;
compiling the computational dataflow graph using a deterministic instruction set architecture that specifies functionality of a special purpose machine learning model processor;
determining an amount of resources required to execute the computational dataflow graph representing the machine learning model; and
allocating resources based on the determined amounts of resources required to execute the machine learning model represented by the computational dataflow graph.
2. The computer-implemented method of claim 1 , wherein determining an amount of resources required to execute the computational data flow graph comprises:
determining a number of operations required for executing the computational dataflow graph;
determining an amount of storage required for executing the computational dataflow graph; and
determining an amount of input/output communications required for executing the computational dataflow graph.
3. The computer-implemented method of claim 1 , wherein scheduling the operations of the computational dataflow graph comprises:
determining a starting clock cycle;
determining a duration for execution of each operation; and
scheduling the operations of the computational dataflow graph using the starting clock cycle and the duration.
4. The method of claim 1 , further comprising:
automatically load-balancing the machine learning model based on operations and input/output required of the model at compile time.
5. The method of claim 1 , wherein the machine learning model executes on the special purpose machine learning model processor in a data center.
6. The method of claim 5 , further comprising:
adjusting data center resources based on the resource use of the machine learning model.
7. The method of claim 1 , wherein allocating resources based on the determined amounts of resources required to execute the machine learning model represented by the computational dataflow graph comprises:
identifying resources for a second machine learning model that executes on the special purpose machine learning model processor, the resources being unused during an allocated time period; and
providing the resources to the machine learning model represented by the computational dataflow graph during the allocated time period until a request for the second machine learning model is received by the special purpose machine learning model processor.
8. The method of claim 1 , further comprising:
rebalancing the machine learning model based on factors including usage, timing, number of operations the model requires, or the precision necessary to perform a machine learning task.
9. A computer-implemented method for allocating resources for machine learning models to run together on a special purpose machine learning model processor, the method comprising:
generating, from machine learning models, computational dataflow graphs, each graph representing one of the machine learning models;
scheduling and compiling, using a deterministic instruction set architecture that specifies functionality of a special purpose machine learning model processor, the computational dataflow graphs representing the machine learning models into executable binaries;
determining an amount of resources required to execute each executable binary;
determining an amount of resources available on the special purpose machine learning model processor;
determining a first executable binary of a computational dataflow graph representing a machine learning model to execute on the special purpose machine learning model processor;
determining a second executable binary of a computational dataflow graph representing a second machine learning model to execute on the special purpose machine learning model processor; and
allocating resources for the first and second machine executable binaries to execute together on the special purpose machine learning model processor.
10. The computer-implemented method of claim 9 , wherein determining an amount of resources required to execute the computational data flow graph comprises:
determining a number of operations required for executing the computational dataflow graph;
determining an amount of storage required for executing the computational dataflow graph; and
determining an amount of input/output communications required for executing the computational dataflow graph.
11. The computer-implemented method of claim 10 , comprising:
determining a starting clock cycle;
determining a duration for execution of each operation; and
scheduling the operations of the computational dataflow graph using the starting clock cycle and the duration.
12. The computer-implemented method of claim 9 , comprising scheduling operations of the computational dataflow graph at least by:
determining a starting clock cycle;
determining a duration for execution of each operation; and
scheduling the operations of the computational dataflow graph using the starting clock cycle and the duration.
13. The computer-implemented method of claim 9 , wherein determining a second executable binary of a computational dataflow graph representing a second machine learning model to execute on the special purpose machine learning model processor comprises:
determining that the second executable binary requires a number of operations that, when added to the number of operations required by the first executable binary, does not exceed a number of operations available on the special purpose machine learning model processor;
determining that the second executable binary requires an amount of storage that, when added to the amount of storage required by the first executable binary, does not exceed an amount of storage available on the special purpose machine learning model processor; and
determining that the second executable binary requires an amount of input/output communications that, when added to the amount of input/output communications required by the first executable binary, does not exceed an amount of input/output communications available on the special purpose machine learning model processor.
14. A system comprising one or more computers and one or more storage devices storing instructions that are executable by the one or more computers to cause the one or more computers to perform the operations of:
receiving a machine learning model to execute on a special purpose machine learning model processor;
generating, from the machine learning model, a computational dataflow graph representing the machine learning model comprising a plurality of nodes, a plurality of connector directed edges, and a plurality of parameter directed edges, wherein each node represents a respective operation performed by the machine learning model as part of the model output from the model input, wherein each connector directed edge connects a respective first node to a respective second node that represents an operation that receives, as input, an output of an operation represented by the respective first node, and wherein each parameter directed edge connects into a respective node and represents a flow of one or more parameters of the model as input to the operation represented by the respective node;
scheduling operations of the computational dataflow graph;
compiling the computational dataflow graph using a deterministic instruction set architecture that specifies functionality of a special purpose machine learning model processor;
determining an amount of resources required to execute the computational dataflow graph representing the machine learning model; and
allocating resources based on the determined amounts of resources required to execute the machine learning model represented by the computational dataflow graph.
15. The system of claim 14 , wherein determining an amount of resources required to execute the computational data flow graph comprises:
determining a number of operations required for executing the computational dataflow graph;
determining an amount of storage required for executing the computational dataflow graph; and
determining an amount of input/output communications required for executing the computational dataflow graph.
16. The system of claim 14 , wherein scheduling the operations of the computational dataflow graph comprises:
determining a starting clock cycle;
determining a duration for execution of each operation; and
scheduling the operations of the computational dataflow graph using the starting clock cycle and the duration.
17. A computer-implemented method for allocating resources for machine learning models to run together on a special purpose machine learning model processor, the method comprising:
generating, from machine learning models, computational dataflow graphs, each graph representing one of the machine learning models;
scheduling and compiling, using a deterministic instruction set architecture that specifies functionality of a special purpose machine learning model processor, the computational dataflow graphs representing the machine learning models into executable binaries;
determining an amount of resources required to execute each executable binary;
determining an amount of resources available on the special purpose machine learning model processor;
determining a first executable binary of a computational dataflow graph representing a machine learning model to execute on the special purpose machine learning model processor;
determine a second executable binary of a computational dataflow graph representing a second machine learning model to execute on the special purpose machine learning model processor, comprising:
determining that the second executable binary requires a number of operations that, when added to the number of operations required by the first executable binary, does not exceed a number of operations available on the special purpose machine learning model processor; and
allocating resources for the first and second machine executable binaries to execute together on the special purpose machine learning model processor.