Neural network processing based on subgraph recognition
Systems and methods for providing executable instructions to a neural network processor are provided. In one example, a system comprises a database that stores a plurality of executable instructions and a plurality of subgraph identifiers, each subgraph identifier of the plurality of subgraph identifiers being associated with a subset of instructions of the plurality of executable instructions. The system further includes a compiler configured to: identify a computational subgraph from a computational graph of a neural network model; compute a subgraph identifier for the computational subgraph, based on whether the subgraph identifier is included in the plurality of subgraph identifiers, either: obtain, from the database, first instructions associated with the subgraph identifier; or generate second instructions representing the computational subgraph; and provide the first instructions or the second instructions for execution by a neural network processor to perform computation operations for the neural network model.
1. A system comprising:
a database that stores subsets of executable instructions, wherein each subset of the subsets of executable instructions is associated with one of a plurality of subgraph identifiers; and
a computer-readable medium having a compiler stored therein that, when executed, is configured to:
identify a computational subgraph from a computational graph of a neural network model representing a sequence of computation operations to be performed for the neural network model, the sequence of computation operations including a plurality of types of computation operations including a convolution operation, an activation function processing operation, or a pooling operation;
determine a subgraph identifier of the computational subgraph;
determine whether the subgraph identifier is included in the plurality of subgraph identifiers stored in the database;
perform one of:
in response to determining that the subgraph identifier is included in the plurality of subgraph identifiers, obtain, from the database, first instructions from one of the subsets of executable instructions associated with the subgraph identifier; or
in response to determining that the subgraph identifier is not included in the plurality of subgraph identifiers, generate second instructions representing the computational subgraph; and
provide the first instructions or the second instructions for execution by a processor.
2. The system of claim 1 , wherein the computational graph includes a plurality of nodes, each node representing one of the plurality of types of computation operations, and wherein each node of the plurality of nodes is connected to an edge which indicates a direction of flow of data to or from the each node and a number of data elements included in the flow of data.
3. The system of claim 2 , wherein the computational subgraph includes a pre-determined subset of the nodes of the computational graph and the edges connected to the subset of nodes, and wherein the computational subgraph represents a part of the sequence of the computation operations;
wherein the computational subgraph is identified based on the subset of the nodes and the edges connected to the subset of nodes; and
wherein the computational graph includes multiple instances of the computational subgraph.
4. The system of claim 3 , wherein the subgraph identifier of the computational subgraph is determined based on:
assigning a node value to each node of the subset of the nodes based on a type of computation operation represented by the each node;
assigning an edge value to each edge of the edges based on the nodes connected to the each edge and a number of data elements in the flow of data represented by the each edge; and
determining the subgraph identifier based on the node values and the edge values.
5. The system of claim 4 , wherein the subgraph identifier is determined based on:
inputting the node values and the edge values to a hash function to compute a hash value; and
determining the subgraph identifier based on the hash value.
6. The system of claim 3 , wherein the first instructions and the second instructions are related to scheduling of resources of the processor to support the part of the sequence of computation operations; and
wherein the processor is configure to perform the part of the sequence of computation operations represented by the computational subgraph at a higher efficiency by executing the first instructions than by executing the second instructions.
7. The system of claim 6 , wherein the first instructions are generated by a machine learning process, a human expert, or both.
8. The system of claim 3 , wherein the second instructions comprise a plurality of primitive operations at the processor, the plurality of primitive operations being generated from decomposing each computation operation included in the computational subgraph.
9. The system of claim 8 , wherein the plurality of primitive operations includes arithmetic operations.
10. The system of claim 1 , wherein the database stores a plurality of processor identifiers and associates the plurality of processor identifiers with the plurality of subgraph identifiers and with the subsets of executable instructions; and
wherein the first instructions are identified based on the subgraph identifier of the computational subgraph and the processor identifier of the processor.
11. The system of claim 1 , wherein the database associates the subgraph identifier of the computational subgraph with first instructions and third instructions; and
wherein the compiler is configured to select the first instructions over the third instructions based on at least one of: execution efficiency or cost.
12. A non-transitory computer-readable medium having instructions which, when executed by one or more processors, cause the one or more processors to execute a compiler, wherein the compiler is configured to:
identify a computational subgraph from a computational graph of a neural network model representing a sequence of computation operations to be performed for the neural network model, the sequence of computation operations including a plurality of types of computation operations including a convolution operation, an activation function processing operation, or a pooling operation;
determine a subgraph identifier of the computational subgraph;
determine whether the subgraph identifier is included in a plurality of subgraph identifiers stored in a database;
perform one of:
in response to determining that the subgraph identifier is included in the plurality of subgraph identifiers, obtain, from the database storing subsets of executable instructions, first instructions from one of the subsets of executable instructions associated with the subgraph identifier; or
in response to determining that the subgraph identifier is not included in the plurality of subgraph identifiers, generate second instructions representing the computational subgraph; and
provide the first instructions or the second instructions for execution by a processor.
13. The non-transitory computer-readable medium of claim 12 , wherein the computational graph includes a plurality of nodes, each node representing one of the plurality of types of computation operations, and wherein each node of the plurality of nodes is connected to an edge which indicates a direction of flow of data to or from the each node and a number of data elements included in the flow of data.
14. The non-transitory computer-readable medium of claim 13 , wherein the computational subgraph includes a pre-determined subset of the nodes of the computational graph and the edges connected to the subset of nodes, and wherein the computational subgraph represents a part of the sequence of the computation operations;
wherein the computational subgraph is identified based on the subset of the nodes and the edges connected to the subset of nodes; and
wherein the computational graph includes multiple instances of the computational subgraph.
15. The non-transitory computer-readable medium of claim 14 , wherein the subgraph identifier of the computational subgraph is determined based on:
assigning a node value to each node of the subset of the nodes based on a type of computation operation represented by the each node;
assigning an edge value to each edge of the edges based on the nodes connected to the each edge and a number of data elements in the flow of data represented by the each edge; and
determining the subgraph identifier based on the node values and the edge values.
16. The non-transitory computer-readable medium of claim 15 , wherein the subgraph identifier is determined based on:
inputting the node values and the edge values to a hash function to compute a hash value; and
determining the subgraph identifier based on the hash value.
17. A method comprising:
identifying a computational subgraph from a computational graph of a neural network model representing a sequence of computation operations to be performed for the neural network model, the sequence of computation operations including a plurality of types of computation operations including a convolution operation, an activation function processing operation, or a pooling operation;
determining a subgraph identifier of the computational subgraph;
determining whether the subgraph identifier is included in a plurality of subgraph identifiers stored in a database;
performing one of:
in response to determining that the subgraph identifier is included in the plurality of subgraph identifiers, obtaining, from the database storing subsets of executable instructions, first instructions from one of the subsets of executable instructions associated with the subgraph identifier; or
in response to determining that the subgraph identifier is not included in the plurality of subgraph identifiers, generating second instructions representing the computational subgraph; and
providing the first instructions or the second instructions for execution by a processor.
18. The method of claim 17 , wherein the computational graph includes a plurality of nodes, each node representing one of the plurality of types of computation operations, and wherein each node of the plurality of nodes is connected to an edge which indicates a direction of flow of data to or from the each node and a number of data elements included in the flow of data.
19. The method of claim 18 , wherein the computational subgraph includes a pre-determined subset of the nodes of the computational graph and the edges connected to the subset of nodes, and wherein the computational subgraph represents a part of the sequence of the computation operations;
wherein the computational subgraph is identified based on the subset of the nodes and the edges connected to the subset of nodes; and
wherein the computational graph includes multiple instances of the computational subgraph.
20. The method of claim 19 , wherein the subgraph identifier of the computational subgraph is determined based on:
assigning a node value to each node of the subset of the nodes based on a type of computation operation represented by the each node;
assigning an edge value to each edge of the edges based on the nodes connected to the each edge and a number of data elements in the flow of data represented by the each edge; and
determining the subgraph identifier based on the node values and the edge values.