Distributing parallelism for parallel processing architectures
View Patent ↗A system comprises a plurality of computation units interconnected by an interconnection network. A method for configuring the system comprises receiving an initial partitioning of instructions into initial subsets corresponding to different portions of a program; forming a refined partitioning of the instructions into refined subsets each including one or more of the initial subsets, including determining whether to combine a first subset and a second subset to form a third subset according to a comparison of a communication cost between the first subset and second subset and a load cost of the third subset that is based at least in part on a number of instructions issued per cycle by a computation unit; and assigning each refined subset of instructions to one of the computation units for execution on the assigned computation unit.
1. A method for configuring a system comprising a plurality of computation units interconnected by an interconnection network, the method comprising:
receiving by a computing system an initial partitioning of instructions into initial subsets corresponding to different portions of a program;
forming by the computing system a refined partitioning of the instructions into refined subsets each including one or more of the initial subsets, including:
determining whether to combine a first subset and a second subset of the initial subsets to form a third subset based at least in part on a comparison of a communication cost between the first subset and second subset to a load cost of the third subset that is based at least in part on a number of instructions issued per cycle by a computation unit; and
assigning each refined subset of instructions to one of the computation units for execution on the assigned computation unit.
2. The method of claim 1 , wherein the load cost of the third subset is based on an estimate of load associated with a number of instructions in the third subset in excess of the number of instructions issued per cycle by a computation unit.
3. The method of claim 2 , wherein the estimate of load comprises an estimate of load on computational resources available in a computation unit.
4. The method of claim 3 , wherein the computational resources available in a computation unit comprise functional units available for concurrent execution of instructions.
5. The method of claim 2 , wherein the estimate of load comprises an estimate of load on storage resources available in a computation unit.
6. The method of claim 5 , wherein the storage resources available in a computation unit comprise registers.
7. The method of claim 1 , wherein the communication cost between the first subset and second subset is based on a number of dependencies between instructions of the first subset and instructions of the second subset.
8. The method of claim 1 , wherein forming the refined partitioning of the instructions into refined subsets comprises repeatedly combining subsets, and determining whether to combine a first subset and a second subset to form a third subset comprises selecting the current smallest subset as the second subset.
9. The method of claim 1 , wherein forming the refined partitioning of the instructions into refined subsets comprises forming an affinity graph based on true data dependence between instructions where an output of one instruction is required as input to another instruction.
10. The method of claim 9 , wherein the initial subsets are leaf nodes of the affinity graph.
11. The method of claim 10 , wherein each non-leaf node in the affinity graph represents a subset of instructions that includes the instructions in the subsets represented by the child nodes of the non-leaf node.
12. The method of claim 10 , wherein the affinity graph has a number of root nodes corresponding to the number of computation units in the system.
13. The method of claim 1 , further comprising forming the initial partitioning of instructions into the initial subsets based on dominant sequence clustering.
14. The method of claim 1 , further comprising forming a specification of the program to be executed by the plurality of computation units based on the assigned refined subsets of instructions.
15. A computer program product tangibly embodied on a computer-readable storage device for compiling a software program for execution on a system comprising a plurality of computation units interconnected by an interconnection network, the computer program product comprising instructions for causing a computer system to:
receive an initial partition of instructions of the software program into initial subsets corresponding to different portions of the software program;
form a refined partition of the instructions of the software program into refined subsets each including one or more of the initial subsets, with the instructions of the computer program product including instructions to:
determine whether to combine a first subset and a second subset to form a third subset based at least in part on a comparison of a communication cost between the first subset and second subset to a load cost of the third subset that is based at least in part on a number of instructions of the software program issued per cycle by a computation unit; and
assign each refined subset of instructions of the software program to one of the computation units for execution on the assigned computation unit.
16. The computer program product of claim 15 wherein the load cost of the third subset is based on an estimate of load associated with a number of instructions in the third subset in excess of the number of instructions issued per cycle by a computation unit.
17. The computer program product of claim 16 wherein the estimate of load comprises an estimate of load on storage resources available in a computation unit.
18. The computer program product of claim 15 wherein the communication cost between the first subset and second subset is based on a number of dependencies between instructions of the first subset and instructions of the second subset.
19. The computer program product of claim 15 wherein instructions to form the refined partition of the instructions into refined subsets comprises instructions to repeatedly combine subsets; and instructions to determine whether to combine a first subset and a second subset to form a third subset comprises instructions to select the current smallest subset as the second subset.
20. The computer program product of claim 15 wherein instructions to form the refined partition of the instructions into refined subsets comprises instructions to form an affinity graph based on true data dependence between instructions where an output of one instruction is required as input to another instruction.
21. The computer program product of claim 20 wherein the initial subsets are leaf nodes of the affinity graph.
22. The computer program product of claim 21 wherein each non-leaf node in the affinity graph represents a subset of instructions that includes the instructions in the subsets represented by the child nodes of the non-leaf node.
23. The computer program product of claim 20 wherein the affinity graph has a number of root nodes corresponding to the number of computation units in the system.
24. The computer program product of claim 15 , further configured to:
form the initial partition of instructions into the initial subsets based on dominant sequence clustering.
25. A system, comprising:
a plurality of interconnected computation devices the computation devices configured to:
receive an initial partition of instructions into initial subsets corresponding to different portions of a program;
form a refined partition of the instructions into refined subsets each including one or more of the initial subsets;
determine whether to combine a first subset and a second subset to form a third subset based at least in part on a comparison of a communication cost between the first subset and second subset to a load cost of the third subset that is based at least in part on a number of instructions issued per cycle by a computation unit; and
assign each refined subset of instructions to one of the computation units for execution on the assigned computation unit.
26. The system of claim 25 , further comprising a memory for storing the information for configuring the computation units.
27. The system of claim 25 , wherein each computation unit comprises
a processor, and
a switch including switching circuitry to forward data received over data paths from other computation units to the processor and to switches of other computation units, and to forward data received from the processor to switches of other computation units.
28. The system of claim 25 wherein the load cost of the third subset is based on an estimate of load associated with a number of instructions in the third subset in excess of the number of instructions issued per cycle by a computation unit.
29. The system of claim 28 wherein the estimate of load comprises an estimate of load on storage resources available in a computation unit.
30. The system of claim 25 wherein the communication cost between the first subset and second subset is based on a number of dependencies between instructions of the first subset and instructions of the second subset.
31. The system of claim 25 wherein the system configured to form the refined partition of the instructions into refined subsets comprises the system configured to repeatedly combine subsets; and determine whether to combine a first subset and a second subset to form a third subset comprises the system configured to select the current smallest subset as the second subset.
32. The system of claim 25 wherein the system configured to form the refined partitioning of the instructions into refined subsets comprises the system configured to form an affinity graph based on true data dependence between instructions where an output of one instruction is required as input to another instruction.
33. The system of claim 32 wherein the initial subsets are leaf nodes of the affinity graph.
34. The system of claim 33 wherein each non-leaf node in the affinity graph represents a subset of instructions that includes the instructions in the subsets represented by the child nodes of the non-leaf node.
35. The system of claim 33 wherein the affinity graph has a number of root nodes corresponding to the number of computation units in the system.
36. The system of claim 25 , further configured to form the initial partition of instructions into the initial subsets based on dominant sequence clustering.