Neural network unit with segmentable array width rotator and re-shapeable weight memory to match segment width to provide common weights to multiple rotator segments
First/second memories hold rows of N weight/data words. The first memory address has log 2 W bits and an extra bit. Each of N processing units (PU) of index J has first and second registers, an accumulator, an arithmetic unit performs an operation thereon to accumulate a result, first multiplexing logic for PUs 0 through (N/2)−1 receives first memory weight words J and J+(N/2) and for PUs N/2 through N−1 receives first memory weight words J and J−(N/2) and outputs a selected weight word to the first register, and second multiplexing logic receives second memory data word J and data word output by the second register of PU J−1 and outputs a selected data word to the second register. PU 0 second multiplexing logic also receives PU (N/2)−1 second register data word, and PU N/2 second multiplexing logic also receives PU N−1 second register data word.
1. An apparatus, comprising:
a first memory that holds rows of N weight words indexed as 0 through N−1 and that receives an address, wherein the address has log 2 W bits and an extra bit;
a second memory that holds rows of N data words indexed as 0 through N−1, wherein N is at least 512;
an array of N processing units (PUs) indexed as 0 through N−1, each PUs of index J having:
first and second registers, an accumulator having an output, and an arithmetic unit having first, second and third inputs and that performs an operation thereon to generate a result to store in the accumulator, the first input receives the output of the accumulator, the second input receives a weight word output by the first register, the third input receives a data word output by the second register;
first multiplexing logic that, for the PUs 0 through (N/2)−1 receives weight words J and J+(N/2) output by the first memory and for PUs N/2 through N−1 receives weight words J and J−(N/2) output by the first memory, and that outputs a selected weight word to the first register; and
second multiplexing logic that receives data word J output by the second memory and receives a data word output by the second register of PU J−1 and that outputs a selected data word to the second register, wherein PU J−1 is PU N−1 for PU 0 ; and
wherein the second multiplexing logic of PU 0 also receives the data word output by the second register of PU (N/2)−1, and the second multiplexing logic of PU N/2 also receives the data word output by the second register of PU N−1.
2. The apparatus of claim 1 , wherein in a first mode, when the address extra bit is a first value of the first multiplexing logic of the PUs 0 through (N/2)−1 selects weight word J for output to the first register and the first multiplexing logic of the PUs N/2 through N−1 selects weight word J−(N/2) for output to the first register, and when the address extra bit is a second value of the first multiplexing logic of the PUs 0 through (N/2)−1 selects weight word J−(N/2) for output to the first register and the first multiplexing logic of the PUs N/2 through N−1 selects weight word J for output to the first register.
3. The apparatus of claim 2 , wherein in the first mode:
when the address extra bit is the first value, the first memory outputs words 0 through (N/2)−1 of a row selected by the address log 2 W bits, but does not output words N/2 through N−1; and
when the address extra bit is the second value, the memory outputs words N/2 through N−1 of a row selected by the address log 2 W bits, but does not output words 0 through (N/2)−1.
4. The apparatus of claim 2 ,
wherein in the first mode, the second multiplexing logic of PU 0 selects the data word output by the second register of PU (N/2)−1 to output to the second register and the second multiplexing logic of PU N/2 selects the data word output by the second register of PU N−1 to output to the second register and the second multiplexing logic of the remaining PUs selects the data word output by the second register of PU J−1 to output to the second register.
5. The apparatus of claim 4 , wherein in the first mode, the second multiplexing logic alternatively selects data word J output by the second memory for output to the second resister.
6. The apparatus of claim 2 , wherein in the first mode the first memory is effectively configured to hold 2*W rows of N/2 words for provision to the PUs 0 through N−1.
7. The apparatus of claim 2 , wherein in a second mode, the first multiplexing logic selects weight word J output by the first memory for output to the first register and the second multiplexing logic selects the data word output by the second register of PU J−1 for output to the second register.
8. The apparatus of claim 7 , wherein in the second mode, the second multiplexing logic alternatively selects data word J output by the second memory for output to the second register.
9. The apparatus of claim 7 , wherein in the second mode, the second multiplexing logic and the second register of the PUs 0 through N−1 collectively operate as an N-word rotater, and
wherein in the first mode, the second multiplexing logic and the second register of the PUs 0 through (N/2)−1 collectively operate as a first N/2-word rotater, and the second multiplexing logic and the second register of the PUs N/2 through N−1 collectively operate as a second N/2 word rotater.
10. A method for operating an apparatus having a first memory holding rows of N weight words indexed as 0 through N−1 and that receive an address, wherein the address has log 2 W bits and an extra bit, a second memory holding rows of N data words indexed as 0 through N−1, wherein N is at least 512, and an array of N processing units (PUs) indexed as 0 through N−1, each PUs of index J having first and second registers, an accumulator having an output, and an arithmetic unit having first, second and third inputs and that performs an operation thereon to generate a result to store in the accumulator, the first input receives the output of the accumulator, the second input receives a weight word output by the first register, the third input receives a data word output by the second register, first multiplexing logic that, for the PUs 0 through (N/2)−1 receives weight words J and J+(N/2) output by the first memory and for the PUs N/2 through N−1 receives weight words J and J−(N/2) output by the first memory, and that outputs a selected weight word to the first register, and second multiplexing logic that receives data word J output by the second memory and receives a data word output by the second register of the PU J−1 and that outputs a selected data word to the second register, wherein PU J−1 is PU N−1 for PU 0 , and wherein the second multiplexing logic of PU 0 also receives the data word output by the second register of PU (N/2)−1, and the second multiplexing logic of PU N/2 also receives the data word output by the second register of the PU N−1, the method comprising:
in a first mode:
when the address extra bit is a first value:
selecting, by the first multiplexing logic of the PUs 0 through (N/2)−1, weight word J for output to the first register; and
selecting, by the first multiplexing logic of the PUs N/2 through N−1, weight word J−(N/2) for output to the first register; and when the address extra bit is a second value:
selecting, by the first multiplexing logic of the PUs 0 through (N/2)−1, weight word J−(N/2) for output to the first register; and
selecting, by the first multiplexing logic of the PUs N/2 through N−1, weight word J for output to the first register.
11. The method of claim 10 , further comprising:
in the first mode:
when the address extra bit is the first value:
outputting, by the first memory, words 0 through (N/2)−1 of a row selected by the address log 2 W bits, but not outputting words N/2 through N−1; and
when the address extra bit is the second value:
outputting, by the memory, words N/2 through N−1 of a row selected by the address log 2 W bits, but not outputting words 0 through (N/2)−1.
12. The method of claim 10 , further comprising:
in the first mode:
selecting, by the second multiplexing logic of the PU 0 , the data word output by the second register of PU (N/2)−1 to output to the second register;
selecting, by the second multiplexing logic of PU N/2, the data word output by the second register of PU N−1 to output to the second register; and
selecting, by the second multiplexing logic of the remaining PUs, the data word output by the second register of PU J−1 to output to the second register.
13. The method of claim 12 , further comprising:
in the first mode:
alternatively selecting, by the second multiplexing logic, data word J output by the second memory for output to the second register.
14. The method of claim 10 , further comprising:
in the first mode:
effectively configuring the first memory to hold 2*W rows of N/2 words for provision to PUs 0 through N−1.
15. The method of claim 10 , further comprising:
in a second mode:
selecting, by the first multiplexing logic, weight word J output by the first memory for output to the first register; and
selecting, by the second multiplexing logic, the data word output by the second register of PU J−1 for output to the second register.
16. The method of claim 15 , further comprising:
in the second mode:
alternatively selecting, by the second multiplexing logic, data word J output by the second memory for output to the second register.
17. The method of claim 15 , further comprising:
in the second mode:
collectively operating, by the second multiplexing logic and the second register of the PUs 0 through N−1, as an N-word rotater; and
in the first mode:
collectively operating, by the second multiplexing logic and the second register of the PUs 0 through (N/2)−1, as a first N/2-word rotater; and
collectively operating, by the second multiplexing logic and the second register of the PUs N/2 through N−1, as a second N/2 word rotater.
18. An apparatus, comprising:
a first memory that holds rows of N weight words indexed as 0 through N−1 and that receives an address, wherein the address has log 2 W bits and an extra bit;
a second memory that holds rows of N data words indexed as 0 through N−1, wherein N is at least 512;
an array of N processing units (PUs) indexed as 0 through N−1, each PU of index J having:
first and second registers, an accumulator having an output, and an arithmetic unit having first, second and third inputs and that performs an operation thereon to generate a result to store in the accumulator, the first input receives the output of the accumulator, the second input receives a weight word output by the first register, the third input receives a data word output by the second register;
first multiplexing logic that, for the PUs 0 through (N/2)−1 receives weight words J and J+(N/2) output by the first memory and for the PUs N/2 through N−1 receives weight words J and J−(N/2) output by the first memory, and that outputs a selected weight word to the first register; and
second multiplexing logic that receives data word J output by the second memory and receives a data word output by the second register of PU J−1 and that outputs a selected data word to the second register, wherein PU J−1 is PU N−1 for PU 0 ; and
wherein the second multiplexing logic of PU N/4 also receives the data word output by the second register of PU (3N/4)−1, and the second multiplexing logic of PU 3N/4 also receives the data word output by the second register of PU (N/4)−1.
19. The apparatus of claim 18 , wherein in a first mode, when the address extra bit is a first value the first multiplexing logic of the PUs 0 through (N/2)−1 selects weight word J for output to the first register and the first multiplexing logic of PUs N/2 through N−1 selects weight word J−(N/2) for output to the first register, and when the address extra bit is a second value the first multiplexing logic of the PUs 0 through (N/2)−1 selects weight word J−(N/2) for output to the first register and the first multiplexing logic of PUs N/2 through N−1 selects weight word J for output to the first register.
20. The apparatus of claim 19 , wherein in the first mode:
when the address extra bit is the first value, the first memory outputs words 0 through (N/2)−1 of a row selected by the address log 2 W bits, but does not output words N/2 through N−1; and
when the address extra bit is the second value, the memory outputs words N/2 through N−1 of a row selected by the address log 2 W bits, but does not output words 0 through (N/2)−1.
21. The apparatus of claim 19 , wherein in the first mode, the second multiplexing logic of PU N/4 selects the data word output by the second register of PU (3N/4)−1 to output to the second register and the second multiplexing logic of PU 3N/4 selects the data word output by the second register of PU (N/4)−1 to output to the second register and the second multiplexing logic of the remaining PUs selects the data word output by the second register of PU J−1 to output to the second register.
22. The apparatus of claim 21 , wherein in the first mode, the second multiplexing logic alternatively selects data word J output by the second memory for output to the second register.
23. The apparatus of claim 19 , wherein in a second mode, the first multiplexing logic selects weight word J output by the first memory for output to the first register and the second multiplexing logic selects the data word output by the second register of PU J−1 for output to the second register.