Graph spatial split
A method for reducing latency and increasing throughput in a reconfigurable computing system includes receiving a compute graph for execution on a reconfigurable dataflow processor comprising a grid of compute units and grid of memory units interconnected with a switching array. The compute graph includes a node specifying an operation on a tensor. The node may be split into multiple nodes that each specify the operation on a distinctive portion of the tensor to produce a first modified compute graph. The first modified compute graph may be executed. In addition, the multiple nodes may be within a single meta-pipeline stage and may be processed in parallel. Furthermore, the compute graph may further comprise a separate node for gathering the distinctive portions of the tensor into a complete tensor, to produce a second modified compute graph.
1. A system for reducing latency and increasing throughput in reconfigurable dataflow processors, the system comprising:
a host computer comprising a graph optimization module configured to conduct a method comprising:
receiving a compute graph for execution on a reconfigurable dataflow computing system, the compute graph comprising a node specifying an operation on a tensor;
splitting the node into multiple nodes that each specify the operation on a distinctive portion of the tensor to produce a first modified compute graph; and
a reconfigurable dataflow processor (RDP) configured to execute the first modified compute graph.
2. The system of claim 1 , wherein splitting the node into X nodes improves latency by a factor of X.
3. The system of claim 1 , wherein the multiple nodes that each specify the operation on the distinctive portion of the tensor are within a single meta-pipeline stage.
4. The system of claim 3 , wherein the multiple nodes that each specify the operation on distinctive portions of the tensor are processed in parallel.
5. The system of claim 3 , further comprising adding a separate node for gathering the distinctive portions of the tensor into a complete tensor, to produce a second modified compute graph.
6. The system of claim 5 , wherein the separate node specifies a concatenation operation, a summation operation, or a tensor assembly operation.
7. The system of claim 5 , wherein the multiple nodes and the separate node are within the single meta-pipeline stage.
8. The system of claim 1 , wherein the RDP comprises a grid of compute units and a grid of memory units interconnected with a switching array, each compute unit comprising an array of arithmetic units organized into I lanes and J meta-pipeline stages.
9. A method for reducing latency and increasing throughput in a reconfigurable computing system, the method comprising:
receiving a compute graph for execution on a reconfigurable dataflow processor (RDP), the compute graph comprising a node specifying an operation on a tensor;
splitting the node into multiple nodes that each specify the operation on a distinctive portion of the tensor to produce a first modified compute graph; and
executing the first modified compute graph on the RDP.
10. The method of claim 9 , wherein splitting the node into X nodes improves latency by a factor of X.
11. The method of claim 9 , wherein the multiple nodes that each specify the operation on the distinctive portion of the tensor are within a single meta-pipeline stage.
12. The method of claim 11 , wherein the multiple nodes that each specify the operation on distinctive portions of the tensor are processed in parallel.
13. The method of claim 11 , further comprising adding a separate node for gathering the distinctive portions of the tensor into a complete tensor, to produce a second modified compute graph.
14. The method of claim 13 , wherein the separate node specifies a concatenation operation, a summation operation, or a tensor assembly operation.
15. The method of claim 13 , wherein the multiple nodes and the separate node are within the single meta-pipeline stage.
16. The method of claim 9 , wherein the RDP comprises a grid of compute units and a grid of memory units interconnected with a switching array, each compute unit comprising an array of arithmetic units organized into I lanes and J meta-pipeline stages.
17. A computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, wherein the program instructions are executable by a processor to cause the processor to conduct a method comprising:
receiving a compute graph for execution on a reconfigurable dataflow processor (RDP), the compute graph comprising a node specifying an operation on a tensor;
splitting the node into multiple nodes that each specify the operation on a distinctive portion of the tensor to produce a first modified compute graph; and
executing the first modified compute graph on the RDP.