Neural network execution streams
Apparatuses, systems, and techniques to perform a neural network. In at least one embodiment, an application programming interface schedules two or more graph nodes to be performed by two or more parallel processing pipelines based, at least in part, on an order of layers in the neural network.
1. A non-transitory machine-readable medium having stored thereon an application programming interface (API), which if performed by one or more processors, cause the one or more processors to at least:
cause two or more neural network graph nodes to be synchronized and performed in parallel by two or more parallel processing pipelines of the one or more processors based, at least in part, on whether two or more neural network layers corresponding to the two or more neural network graph nodes are dependent on each other, wherein each node of the two or more neural network graph nodes comprises one or more operations to be performed by one or more execution streams associated with the two or more parallel processing pipelines.
2. The non-transitory machine-readable medium of claim 1 , wherein the one or operations correspond to one or more basic blocks.
3. The non-transitory machine-readable medium of claim 1 , wherein the one or more operations are to be performed in series.
4. The non-transitory machine-readable medium of claim 1 , wherein the two or more neural network graph nodes are performed based at least in part on dividing a graph into portions and executing each portion on a respective execution stream of the one or more execution streams.
5. The non-transitory machine-readable medium of claim 4 , wherein a graph node is assigned to a portion of the graph based at least in part on a distance of the graph node to an exit of the graph.
6. The non-transitory machine-readable medium of claim 5 , wherein the distance is based at least in part on an estimated cost of performing an operation associated with the graph node.
7. The non-transitory machine-readable medium of claim 5 , wherein the graph node is determined to be a candidate for assignment to the portion when an in-degree of the graph node is zero.
8. The non-transitory machine-readable medium of claim 1 , wherein execution of a first basic block by a first of the two or more neural network graph nodes to be performed in parallel is synchronized with execution of a second basic block by a second of the two or more neural network graph nodes to be performed in parallel.
9. One or more processors, comprising:
one or more arithmetic logic units (ALUs) to be configured to cause two or more neural network graph nodes to be synchronized and performed in parallel by two or more parallel processing pipelines of the one or more processors based, at least in part, on whether two or more neural network layers corresponding to the two or more neural network graph nodes are dependent on each other, wherein each node of the two or more neural network graph nodes comprises one or more operations to be performed by one or more execution streams associated with the two or more parallel processing pipelines.
10. The one or more processors of claim 9 , wherein a graph is divided into portions, and wherein each portion is executed by a respective execution stream of the one or more execution streams associated with each one of the two or more neural network graph nodes to be performed in parallel.
11. The one or more processors of claim 10 , wherein the one or more execution streams are associated with a plurality of operations to execute in series.
12. The one or more processors of claim 10 , wherein a graph node is assigned to a portion of the graph based at least in part on a distance of the graph node to an exit of the graph.
13. The one or more processors of claim 10 , wherein the graph is divided into the portions based at least in part on assignment of graph nodes associated with a basic block to a first execution stream of the one or more execution streams.
14. The one or more processors of claim 9 , wherein a graph node corresponds to an operation of a neural network comprising the two or more neural network graph nodes, and wherein an edge of a graph corresponds to data flow between operations.
15. The one or more processors of claim 9 , wherein each of the two or more neural network graph nodes to be performed in parallel is to execute operations associated with the one or more execution streams.
16. The one or more processors of claim 15 , wherein a node of a graph is added to a list of candidates for assigning to the one or more execution streams when an in-degree of the node is zero.
17. The one or more processors of claim 15 , wherein operations associated with a basic block are executed by the one or more execution streams.
18. A system, comprising:
one or more processors to be configured to at least cause two or more neural network graph nodes to be synchronized and performed in parallel by two or more parallel processing pipelines of the one or more processors based, at least in part, on whether two or more neural network layers corresponding to the two or more neural network graph nodes are dependent on each other, wherein each node of the two or more neural network graph nodes comprises one or more operations to be performed by one or more execution streams associated with the two or more parallel processing pipelines.
19. The system of claim 18 , wherein the one or operations correspond to one or more basic blocks.
20. The system of claim 18 , wherein the one or more operations are to be performed in series.
21. The system of claim 18 , wherein the two or more neural network graph nodes are to be performed based at least in part on division of a graph into portions and execution of the portions by the one or more execution streams.
22. The system of claim 21 , wherein division of the graph into the portions comprises assignment of a graph node to a portion of the graph based at least in part on a distance of the graph node to an exit of the graph.
23. The system of claim 22 , wherein the distance is based at least in part on an estimated cost of performing an operation associated with the graph node.
24. The system of claim 21 , wherein a graph node is determined to be a candidate for assignment to the portion of the graph when an in-degree of a graph node is zero.
25. The system of claim 18 , wherein execution of a first basic block by a first of the two or more neural network graph nodes to be performed in parallel is synchronized with execution of a second basic block by a second of the two or more neural network graph nodes to be performed in parallel.
26. A system, comprising:
one or more processors to be configured to perform a neural network by at least receiving a graph definition and causing two or more neural network graph nodes of a graph to be synchronized and performed in parallel by two or more parallel processing pipelines of the one or more processors based, at least in part, on whether two or more neural network layers corresponding to the two or more neural network graph nodes are dependent on each other, wherein each node of the two or more neural network graph nodes comprises one or more operations to be performed by one or more execution streams associated with the two or more parallel processing pipelines.
27. The system of claim 26 , wherein the one or operations correspond to one or more basic blocks.
28. The system of claim 26 , wherein the neural network is performed based at least in part on dividing the graph into portions and executing each portion on a respective execution stream of the one or more execution streams.
29. The system of claim 28 , wherein a graph node is assigned to a portion of the graph based at least in part on a distance of the graph node to an exit of the graph.
30. The system of claim 29 , wherein the distance is based at least in part on an estimated cost of performing an operation associated with the graph node.
31. The system of claim 28 , wherein a graph node is determined to be a candidate for assignment to a portion of the graph when an in-degree of the graph node is zero.
32. The system of claim 28 , wherein execution of a first basic block by a first of the two or more neural network graph nodes to be performed in parallel is synchronized with execution of a second basic block by a second of the two or more neural network graph nodes to be performed in parallel.