Seamless place and route for heterogeneous network of processor cores
Methods and systems related to parallel computing using heterogeneous networks of computational nodes are disclosed herein. A method for executing a complex computation on a heterogeneous set of computational nodes linked together by a set of links in a network is disclosed. The method includes compiling, using a table of bandwidth values for the set of links in the network, a set of instructions for routing data for the execution of the complex computation. The method also includes configuring a set of programmable controllers on the heterogeneous set of computational nodes with the set of instructions. The method also includes executing the set of instructions using the set of programmable controllers. The method also includes routing data through the network to facilitate the execution of the complex computation by the heterogeneous set of computational nodes and in response to the execution of the instructions.
1. A method, for executing a complex computation on a heterogeneous set of computational nodes linked together by a set of links in a network, comprising:
compiling, using a table of bandwidth values for the set of links in the network, a set of instructions for routing data for an execution of the complex computation, the set of instructions for routing data and a set of instructions for the execution of the complex computation being generated prior to the execution of the complex computation;
configuring a set of programmable controllers on the heterogeneous set of computational nodes with the set of instructions for routing data;
executing the set of instructions for routing data using the set of programmable controllers; and
routing the data: (i) through the network; (ii) to facilitate the execution of the complex computation by the heterogeneous set of computational nodes; and (iii) in response to the execution of the set of instructions for routing data.
2. The method of claim 1 , wherein:
the complex computation is a directed graph;
the directed graph implements a neural network; and
the data routed through the network is a set of neural network data.
3. The method of claim 1 , wherein:
the heterogeneous set of computational nodes includes a set of processing cores in at least two multicore processors.
4. The method of claim 3 , wherein:
the network includes a network hierarchy with at least a core level and a chip level;
each computational node in the heterogeneous set of computational nodes includes a router from a set of routers; and
the routing of the data through the network includes the set of routers transitioning the data through the core level and the chip level.
5. The method of claim 1 , wherein:
the set of instructions for routing data routes the data using a set of packet identifiers for a set of packets;
each computational node in the heterogeneous set of computational nodes includes a router from a set of routers; and
the routers in the set of routers route the data using the set of packet identifiers.
6. The method of claim 5 , wherein:
the network includes a network hierarchy with at least a core level and a chip level; and
the data is routed through the core level and the chip level using the set of packet identifiers.
7. The method of claim 5 , wherein:
at least one of the links in the set of links is an ethernet link between a first multicore processor chip and a second multicore processor chip; and
a router in the set of routers routes the data through the ethernet link.
8. The method of claim 1 , wherein:
the compiling also uses a table of latency values for the set of links in the network.
9. The method of claim 1 , wherein:
the compiling of the set of instructions for routing data uses a source code description of the complex computation;
the network includes a network hierarchy with at least a core level and a chip level; and
the source code description does not include any information related to the network hierarchy.
10. The method of claim 9 , wherein:
the network hierarchy includes a server level; and
at least one of the links in the set of links is an ethernet link.
11. The method of claim 1 , wherein:
the compiling of the set of instructions for routing data uses a source code description of the complex computation; and
a set of cores identified in the source code description do not correlate with a set of computational nodes in the heterogeneous set of computational nodes.
12. A method for executing a complex computation on a heterogeneous set of computational nodes linked together by a set of links in a network comprising:
compiling, using a machine model of the set of links, a set of instructions for routing data for an execution of the complex computation, wherein the machine model includes a bandwidth for each link in the set of links in the network, the set of instructions for routing data and a set of instructions for the execution of the complex computation being generated prior to the execution of the complex computation;
configuring a set of programmable controllers on the heterogeneous set of computational nodes with the set of instructions for routing data;
executing the set of instructions for routing data using the set of programmable controllers; and
routing data: (i) through the network; (ii) to facilitate the execution of the complex computation by the heterogeneous set of computational nodes; and (iii) in response to the execution of the set of instructions for routing data.
13. The method of claim 12 , wherein:
the heterogeneous set of computational nodes includes a set of processing cores in at least two multicore processors.
14. The method of claim 12 , wherein:
the machine model includes a table of latency values and a table of bandwidth values for the set of links in the network.
15. The method of claim 12 , wherein:
the compiling of the set of instructions for routing data uses a source code description of the complex computation;
the network includes a network hierarchy with at least a core level and a chip level; and
the source code description does not include any information related to the network hierarchy.
16. The method of claim 15 , wherein:
the network hierarchy includes a server level and a rack level; and
at least one of the links in the set of links is an ethernet link.
17. The method of claim 12 , wherein:
the compiling of the set of instructions for routing data uses a source code description of the complex computation; and
a set of cores identified in the source code description do not correlate with a set of computational nodes in the heterogeneous set of computational nodes.
18. The method of claim 12 , wherein:
the network includes a network hierarchy with at least a core level and a chip level;
each computational node in the heterogeneous set of computational nodes includes a router from a set of routers; and
the routing of the data through the network includes the set of routers transitioning the data through the core level and the chip level.
19. The method of claim 12 , wherein:
the set of instructions for routing data routes the data using a set of packet identifiers for a set of packets;
each computational node in the heterogeneous set of computational nodes includes a router from a set of routers; and
the routers in the set of routers route the data using the set of packet identifiers.
20. The method of claim 19 , wherein:
the network includes a network hierarchy with at least a core level and a chip level; and
the data is routed through the core level and the chip level using the set of packet identifiers.
21. A system for executing a directed graph, comprising:
a heterogeneous set of computational nodes;
a set of links in a network, wherein the set of links link the computational nodes in the heterogeneous set of computational nodes;
a compiler configured to compile, using a table of bandwidth values for the set of links in the network, a set of instructions for routing data for an execution of the directed graph, the set of instructions for routing data and a set of instructions for the execution of the directed graph being generated prior to the execution of the directed graph; and
a set of programmable controllers on the heterogeneous set of computational nodes configured with the set of instructions for routing data;
wherein executing the set of instructions for routing data routes data through the network to facilitate the execution of the directed graph by the heterogeneous set of computational nodes.
22. The system of claim 21 , wherein:
the directed graph is a neural network; and
the heterogeneous set of computational nodes includes a neural network accelerator and a graphics processing unit.