Machine learning network implemented by statically scheduled instructions, with MLA chip
A compiler receives a description of a machine learning network and generates a computer program that implements the machine learning network. The computer program includes statically scheduled instructions that are executed by a mesh of processing elements (Tiles). The instructions executed by the Tiles are statically scheduled because the compiler can determine which instructions are executed by which Tiles at what times. For example, for the statically scheduled instructions, there are no conditions, branching or data dependencies that can be resolved only at run-time, and which would affect the timing and order of the execution of the instructions.
1. A system comprising a machine learning accelerator (MLA) implemented on a semiconductor die, the MLA comprising:
an on-chip memory system configured to store data used in implementing computations in a machine learning network, the on-chip memory system comprising SRAM; and
a mesh of interconnected Tiles coupled to the on-chip memory system, wherein a computer program generated by a compiler implements the machine learning network on the MLA and the computer program comprises deterministic phases of Tile instructions that implement the computations in the machine learning network and non-deterministic phases that include instructions for data transfer between the on-chip memory system and an off-chip memory comprising DRAM that is not on the same semiconductor die as the on-chip memory system, said instructions in the non-deterministic phases executed by a controller external to the mesh of Tiles; and the Tiles configured to execute the deterministic phases of Tile instructions that implement the computations in the machine learning network; the Tile instructions including (a) Tile compute instructions for performing the computations in the machine learning network, and (b) Tile data transfer instructions for data transfer along data transfer paths between different Tiles and between Tiles and the on-chip memory system; and
wherein each deterministic phase utilizes multiple Tiles with concurrent execution of Tile compute instructions and Tile data transfer instructions by different Tiles, and the Tiles are configured to execute Tile instructions in a same deterministic phase in synchronization and the Tile instructions within each deterministic phase are statically scheduled relative to the other Tile instructions in the same deterministic phase with unconditional start times for every Tile instruction within the same deterministic phase so that the Tiles execute the Tile instructions without any run-time determination of whether the data, Tiles or data transfer paths required for the Tile instructions are available.
2. The system of claim 1 wherein the Tiles are configured to execute the Tile data transfer instructions without circuitry for performing run-time arbitration for data transfer paths to and from the Tiles.
3. The system of claim 1 wherein the Tiles are configured to execute the Tile data transfer instructions without circuitry for run-time checking or resolving contentions or collisions on data transfer paths to and from the Tiles.
4. The system of claim 1 wherein the Tiles are configured to execute the Tile data transfer instructions without circuitry for run-time routing for the data transfers.
5. The system of claim 1 wherein the Tile compute instructions execute on compute pipelines within the Tiles, and the Tiles are configured to execute the Tile compute instructions without circuitry for determining run-time availability of the compute pipelines.
6. The system of claim 1 wherein the Tiles are configured to execute the Tile instructions in each deterministic phase, without circuitry for determining run-time conditions or availability of Tiles used to execute the Tile instructions.
7. The system of claim 1 wherein the MLA further comprises:
circuitry external to the mesh of Tiles but coupled to the mesh of Tiles; the circuitry configured to, for each deterministic phase, synchronize the Tiles utilized in that deterministic phase at a beginning of that deterministic phase.
8. The system of claim 1 wherein the MLA further comprises:
circuitry within the mesh of Tiles configured to, for each deterministic phase, synchronize the Tiles utilized in that deterministic phase at a beginning of that deterministic phase.
9. The system of claim 1 wherein the on-chip memory system is a multi-level memory system that includes L1 memories, L2 memories and data transfer paths between the L1 memories and between the L1 and L2 memories; and the Tile data transfer instructions includes Tile instructions for data transfer between the L1 memories and between the L1 and L2 memories.
10. The system of claim 9 wherein only Tiles on an edge of the mesh have direct access to the L2 memories.
11. The system of claim 1 wherein the Tile instructions include instructions to configure the on-chip memory system as registers for Tiles that perform computations.
12. The system of claim 1 wherein the MLA comprises two or more meshes of interconnected Tiles and further comprises an on-chip interconnect that provides data transfer between different meshes of interconnected Tiles.
13. The system of claim 1 wherein the Tiles have parallel pipelines for execution of Tile data transfer instructions and Tile compute instructions.
14. The system of claim 1 wherein the mesh of Tiles includes at least two different types of Tiles.
15. The system of claim 14 wherein the mesh of Tiles includes Tiles that perform only data transfer by executing Tile data transfer instructions and do not perform computations.
16. The system of claim 1 wherein data transfer paths between Tiles in the mesh establish a preferred direction for data flow through the Tiles.