Methods and circuits for streaming data to processing elements in stacked processor-plus-memory architecture
A stacked processor-plus-memory device includes a processing die with an array of processing elements of an artificial neural network. Each processing element multiplies a first operand—e.g. a weight—by a second operand to produce a partial result to a subsequent processing element. To prepare for these computations, a sequencer loads the weights into the processing elements as a sequence of operands that step through the processing elements, each operand stored in the corresponding processing element. The operands can be sequenced directly from memory to the processing elements or can be stored first in cache. The processing elements include streaming logic that disregards interruptions in the stream of operands.
1. An integrated circuit (IC) device comprising:
a memory die having memory to store data, including a first operand; and
a processor die bonded to the memory die and having:
a temporally continuous connection to the memory die, the temporally continuous connection to convey the first operand from the memory die;
a first processing element having:
a first processing-element input port to receive the first operand and a second operand;
first processing-element storage connected to the memory die via the temporally continuous connection, the first processing-element storage to store the first operand received uninterrupted from the memory die; and
a first processor coupled to the first processing-element storage, the first processor to produce a first processing-element output that is a function of the first operand and the second operand; and
a second processing element having:
second processing-element storage connected to the memory die via the first processing element and the temporally continuous connection, the second processing-element storage to store a third operand received from the memory die, wherein the third operand received from the memory die is interrupted by the first processing element.
2. The IC device of claim 1 , the second processing element having a second processor coupled to the second processing-element storage and the first processing element, the second processor to produce a second processing-element output that is a function of the third operand and the first processing-element output.
3. The IC device of claim 1 , the processor die further comprising a selectable buffer coupled between the processing-element storage and the temporally continuous connection to the memory die, the selectable buffer to store a third operand from the memory die and provide the third operand to the first processing-element storage after a delay.
4. The IC device of claim 3 , the processor die further comprising a sequencer to stream the third operand in a stream of operands to the first processing element, and wherein the selectable buffer stores the third operand for a period of the stream of operands.
5. The IC device of claim 4 , wherein the stream of operands includes an interruption of at least one period of the stream of operands, the first processing element further comprising streaming logic to disregard the interruption.
6. The IC device of claim 1 , the processor die further including:
a second processing element having:
second processing-element storage connected to the memory die via the first processing element, the second processing-element storage to store a third operand received from the memory die via the first processing element;
a second processing-element input port to receive the first processing-element output; and
a second processor coupled to the second processing-element storage and the second processing-element input port, the second processor to produce a second processing-element output that is a function of the third operand and the first processing-element output.
7. The IC device of claim 1 , the memory die to convey the first operand as one of a sequence of operands, the sequence of operands including consecutive operands separated from adjacent operands by a streaming period and gapped operands separated from adjacent operands by a timing gap greater than the streaming period.
8. The IC device of claim 7 , the first processing element including streaming logic to disregard the timing gap.
9. The IC device of claim 8 , the streaming logic to count the operands in the sequence of operands and store the first operand in the first processing-element storage responsive to the count.
10. The IC device of claim 9 , the streaming logic to modify the count and pass the modified count to a second processing element.
11. A method of streaming operands from a memory die into a series of processing elements, each processing element having local processing-element storage, the method comprising:
initializing each of the processing elements to store a skip number;
sequencing the operands through the processing elements, the operands separated by at least one streaming period;
changing the skip number at each processing element for each of the operands presented to the processing element; and
at each of the processing elements presented with a current one of the operands:
comparing the skip number to a reference; and
if the skip number matches the reference, storing the current one of the operands in the processing-element storage.
12. The method of claim 11 , further comprising, at each of the processing elements presented with the current one of the operands, passing the current one of the operands to a downstream one of the processing elements if the skip number does not match the reference.
13. The method of claim 11 , wherein the reference is the same for each of the processing elements.
14. The method of claim 11 , wherein the initializing of the processing element to store the skip numbers comprises passing a first skip number to a first of the processing elements, changing the first skip number to a second skip number, and passing the second skip number to a downstream one of the processing elements.
15. The method of claim 11 , wherein a first of the operands follows a second of the operands by a gap of a multiple of the streaming period.
16. The method of claim 15 , further comprising streaming a valid signal through the processing elements, the streaming asserting the valid signal at each of the processing elements for the first of the operands and the second of the operands and deasserting the valid signal for the gap.
17. The method of claim 16 , further comprising, at each of the processing elements, storing the current one of the operands in the processing element if the skip number matches the reference and the valid signal is asserted and ignoring another of the operands if the skip number matches the reference and the valid signal is deasserted.
18. A method of streaming operands from a memory die into a series of processing elements, each processing element having local processing-element storage, the method comprising:
initializing each of the processing elements with a respective stored operand identifier;
sequencing the operands through the processing elements, each operand paired with a corresponding streamed operand identifier; and
at each of the processing elements presented with one of the operands:
comparing the stored operand identifier with the streamed operand identifier; and
if the stored operand identifier matches the streamed operand identifier, storing the one of the operands paired with the matching streamed operand identifier.