Deterministic memory for tensor streaming processors
Embodiments are directed to a deterministic streaming system with one or more deterministic streaming processors each having an array of processing elements and a first deterministic memory coupled to the processing elements. The deterministic streaming system further includes a second deterministic memory with multiple data banks having a global memory address space, and a controller. The controller initiates retrieval of first data from the data banks of the second deterministic memory as a first plurality of streams, each stream of the first plurality of streams streaming toward a respective group of processing elements of the array of processing elements. The controller further initiates writing of second data to the data banks of the second deterministic memory as a second plurality of streams, each stream of the second plurality of streams streaming from the respective group of processing elements toward a respective data bank of the second deterministic memory.
1. A deterministic streaming system comprising:
one or more deterministic streaming processors, each deterministic streaming processor including:
an array of processing elements, and
a first deterministic memory communicatively coupled to the array of processing elements;
a second deterministic memory communicatively coupled with the array of processing elements, the second deterministic memory comprising a plurality of data banks having a global memory address space for the one or more deterministic streaming processors, wherein the first deterministic memory comprises a static memory and the second deterministic memory comprises a dynamic memory; and
a controller communicatively coupled with the second deterministic memory and configured to:
initiate retrieval of first data from the plurality of data banks of the second deterministic memory as a first plurality of streams, each stream of the first plurality of streams streaming toward a respective group of processing elements of the array of processing elements, and
initiate writing of second data to the plurality of data banks of the second deterministic memory as a second plurality of streams, each stream of the second plurality of streams streaming from the respective group of processing elements toward a respective data bank of the second deterministic memory.
2. The deterministic streaming system of claim 1 , wherein the controller is further configured to assign each stream of the first plurality of streams and the second plurality of streams to a respective pair of pseudo channels of a plurality of pseudo channels of the second deterministic memory.
3. The deterministic streaming system of claim 2 , wherein the respective pair of pseudo channels form an address channel of the second deterministic memory.
4. The deterministic streaming system of claim 2 , wherein each pair of pseudo channels of the plurality of pseudo channels is mapped to the respective group of processing elements.
5. The deterministic streaming system of claim 1 , wherein the second deterministic memory is configured for simultaneous access of the plurality of data banks.
6. The deterministic streaming system of claim 1 , wherein the first deterministic memory is communicatively coupled to each group of processing elements in the array of processing elements via a respective superlane of the deterministic streaming processor.
7. The deterministic streaming system of claim 1 , wherein:
each stream of the first plurality of streams is configured to stream toward the respective group of processing elements via a respective superlane of the deterministic streaming processor; and
each stream of the second plurality of streams is configured to stream toward the second deterministic memory via the respective superlane.
8. The deterministic streaming system of claim 1 , wherein the dynamic memory comprises one or more three-dimensional stacks of a high bandwidth memory devices.
9. The deterministic streaming system of claim 1 , wherein the static memory is a static random access memory, and the dynamic memory comprises one or more three-dimensional stacks of a dynamic random access memory devices.
10. The deterministic streaming system of claim 1 , wherein:
the one or more deterministic streaming processors comprise a plurality of deterministic streaming processors organized as a node of the deterministic streaming system; and
the controller is further configured to:
assign each of the deterministic streaming processors to a respective pair of pseudo channels of a plurality of pseudo channels of the second deterministic memory, and
initiate streaming of data between each of the deterministic streaming processors and a respective data bank of the plurality of data banks of the second deterministic memory associated with the respective pair of pseudo channels.
11. The deterministic streaming system of claim 1 , wherein:
the deterministic streaming system further comprises a plurality of nodes organized as a rack of the deterministic streaming system, each of the nodes comprising a plurality of deterministic streaming processors; and
the controller is further configured to:
assign each of the nodes to a respective pair of pseudo channels of a plurality of pseudo channels of the second deterministic memory, and
initiate streaming of data between each of the nodes and a respective data bank of the plurality of data banks of the second deterministic memory associated with the respective pair of pseudo channels.
12. The deterministic streaming system of claim 1 , wherein:
the deterministic streaming system further comprises a plurality of racks, each of the racks comprises a plurality of nodes, each of the nodes comprising a plurality of deterministic streaming processors; and
the controller is further configured to:
assign each of the racks to a respective pair of pseudo channels of a plurality of pseudo channels of the second deterministic memory, and
initiate streaming of data between each of the racks and a respective data bank of the plurality of data banks of the second deterministic memory associated with the respective pair of pseudo channels.
13. The deterministic streaming system of claim 1 , further comprising a compiler configured to:
manage initiation of the retrieval of the first data at a predetermined first time instant; and
manage initiation of the writing of the second data at a predetermined second time instant.
14. A method of deterministic computing at a deterministic streaming system, the method comprising:
initiating streaming of data between a first deterministic memory of a deterministic streaming processor of the deterministic streaming system and an array of processing elements of the deterministic streaming processor;
initiating retrieval of first data from a plurality of data banks of a second deterministic memory of the deterministic streaming system as a first plurality of streams, each stream of the first plurality of streams streaming toward a respective group of processing elements of the array of processing elements, the second deterministic memory having a global memory address space for one or more deterministic streaming processors of the deterministic streaming system; and
initiating writing of second data to the plurality of data banks as a second plurality of streams, each stream of the second plurality of streams streaming from the respective group of processing elements toward a respective data bank of the second deterministic memory,
wherein the one or more deterministic streaming processors comprise a plurality of deterministic streaming processors organized as a node of the deterministic streaming system, and the method further comprising:
assigning each of the deterministic streaming processors to a respective pair of pseudo channels of a plurality of pseudo channels of the second deterministic memory; and
initiating streaming of data between each of the deterministic streaming processors and a respective data bank of the plurality of data banks of the second deterministic memory associated with the respective pair of pseudo channels.
15. The method of claim 14 , further comprising:
assigning each stream of the first plurality of streams and the second plurality of streams to a respective pair of pseudo channels of the plurality of pseudo channels of the second deterministic memory; and
mapping each pair of pseudo channels of the plurality of pseudo channels to the respective group of processing elements.
16. The method of claim 14 , further comprising:
initiating streaming of each stream of the first plurality of streams toward the respective group of processing elements via a respective superlane of the deterministic streaming processor; and
initiating streaming of each stream of the second plurality of streams toward the second deterministic memory via the respective superlane.
17. The method of claim 14 , wherein a plurality of nodes, including the node, are organized as a rack of the deterministic streaming system.
18. A non-transitory computer-readable storage medium having stored thereon executable instructions, which when executed by a computer processor cause the computer processor to:
initiate streaming of data between a first deterministic memory of a deterministic streaming processor of a deterministic streaming system and an array of processing elements of the deterministic streaming processor;
initiate retrieval of first data from a plurality of data banks of a second deterministic memory of the deterministic streaming system as a first plurality of streams, each stream of the first plurality of streams streaming toward a respective group of processing elements of the array of processing elements, the second deterministic memory having a global memory address space for one or more deterministic streaming processors of the deterministic streaming system; and
initiate writing of second data to the plurality of data banks as a second plurality of streams, each stream of the second plurality of streams streaming from the respective group of processing elements toward a respective data bank of the second deterministic memory,
wherein the one or more deterministic streaming processors comprise a plurality of deterministic streaming processors organized as a node of the deterministic streaming system, and the method further comprising:
assigning each of the deterministic streaming processors to a respective pair of pseudo channels of a plurality of pseudo channels of the second deterministic memory; and
initiating streaming of data between each of the deterministic streaming processors and a respective data bank of the plurality of data banks of the second deterministic memory associated with the respective pair of pseudo channels.
19. The non-transitory computer-readable storage medium of claim 18 , wherein a plurality of nodes, including the node, are organized as a rack of the deterministic streaming system.
20. The non-transitory computer-readable storage medium of claim 18 , wherein the instructions, when expected by the computer processor further cause the computer processor to: further comprising:
initiate streaming of each stream of the first plurality of streams toward the respective group of processing elements via a respective superlane of the deterministic streaming processor;
and initiate streaming of each stream of the second plurality of streams toward the second deterministic memory via the respective superlane.