Neural network unit with neural memory and array of neural processing units that collectively perform multi-word distance rotates of row of data received from neural memory

1. An apparatus, comprising:

an array of N processing units (PU) each having:

an accumulator having an output;

an arithmetic unit having first, second and third inputs and that performs an operation thereon to generate a result to store in the accumulator, the first input receives the output of the accumulator;

a weight input that is received by the second input to the arithmetic unit; and

a multiplexed register having first, second, third and fourth data inputs, an output received by the third input to the arithmetic unit, and a control input that controls selection of the first, second, third and fourth data inputs;

a first memory that holds rows of N weight words and provides the N weight words of a row to the corresponding weight inputs of the N PUs of the PU array;

a second memory that holds rows of N data words and provides the N data words of a row to the corresponding first data inputs of the multiplexed register of the N PUs of the PU array;

wherein the output of the multiplexed register is also received by:

the second data input of the multiplexed register of a PU one PU away;

the third data input of the multiplexed register of a PU 2{circumflex over ( )} J PUs away, wherein J is an integer greater than 1; and

the fourth data input of the multiplexed register of a PU 2{circumflex over ( )}K PUs away, wherein K is an integer greater than J;

wherein the multiplexed registers of the N PUs collectively operate as an N-word rotater that rotates by one word when the control input specifies the second data input;

wherein the multiplexed registers of the N PUs collectively operate as an N-word rotater that rotates by 2{circumflex over ( )}J words when the control input specifies the third data input; and

wherein the multiplexed registers of the N PUs collectively operate as an N-word rotater that rotates by 2{circumflex over ( )}K words when the control input specifies the fourth data input.

2. The apparatus of claim 1 ,

wherein the array of N PUs are configured to execute instructions of a program that 3-dimensionally convolves a W column by H row by C channel input with F of S column by R row by C channel filters to generate F of P column by Q row outputs; and

wherein the program includes instructions that cause:

(a) S times, the control input to select the second data input and the arithmetic unit to multiply the second and third inputs to generate a product and to accumulate the product with the first input to generate the result; and

(b) one or more times, the control input to select the third or fourth data input to accomplish rotating by B minus S words, wherein B is the smallest factor of N that is at least as great as W.

3. The apparatus of claim 2 ,

wherein the program includes instructions that further cause:

(c) S times concurrent with operation (a), the first memory to provide the N weight words of a row to the corresponding weight inputs of the N PUs of the PU array.

4. The apparatus of claim 3 ,

wherein the program includes instructions that further cause:

(d) operations (a), (b) and (c) to be performed at least C times.

5. The apparatus of claim 4 ,

wherein the program includes instructions that further cause:

(e) prior to operation (d), the second memory to provide the N data words of a row to the corresponding first data inputs of the multiplexed register of the N PUs of the PU array.

6. The apparatus of claim 5 ,

wherein the row provided by the second memory contains a horizontal slice of the input corresponding to one of the H rows of the input.

7. The apparatus of claim 6 ,

wherein each of the S times C rows provided by the first memory are logically partitioned as F filter blocks, wherein each filter block of the F filter blocks contains P copies of a different weight from a corresponding one of the F filters.

8. The apparatus of claim 7 ,

wherein a location within the one of the F filters of the different weight is determined at least in part by which of the S times operations (a) and (c) are performed and which of the at least C times operation (d) is performed.

9. The apparatus of claim 1 ,

wherein to perform the operation on the first, second and third inputs, the arithmetic unit multiplies the second and third inputs to generate a product and accumulates the product with the first input to generate the result.

10. The apparatus of claim 1 ,

wherein the multiplexed register of each PU of the array of N PUs further has a fifth data input, and the control input controls selection of the first, second, third, fourth and fifth data inputs;

wherein the output of the multiplexed register is also received by:

the fifth data input of the multiplexed register of a PU 2{circumflex over ( )}L PUs away, wherein L is an integer greater than K; and

wherein the multiplexed registers of the N PUs collectively operate as an N-word rotater that rotates by 2{circumflex over ( )}L words when the control input specifies the fifth data input.

11. A method for operating an apparatus comprising an array of N processing units (PU) each having an accumulator having an output, an arithmetic unit having first, second and third inputs and that performs an operation thereon to generate a result to store in the accumulator, wherein the first input receives the output of the accumulator, the array of PUs further having a weight input that is received by the second input to the arithmetic unit, and a multiplexed register having first, second, third and fourth data inputs, an output received by the third input to the arithmetic unit, and a control input that controls selection of the first, second, third and fourth data inputs, the apparatus further comprising a first memory that holds rows of N weight words and provides the N weight words of a row to the corresponding weight inputs of the N PUs of the PU array and a second memory that holds rows of N data words and provides the N data words of a row to the corresponding first data inputs of the multiplexed register of the N PUs of the PU array, wherein the output of the multiplexed register is also received by the second data input of the multiplexed register of a PU one PU away, the third data input of the multiplexed register of a PU 2{circumflex over ( )}J PUs away, wherein J is an integer greater than 1, and the fourth data input of the multiplexed register of a PU 2{circumflex over ( )}K PUs away, wherein K is an integer greater than J, the method comprising:

collectively operating the multiplexed registers of the N PUs as an N-word rotater that rotates by one word when the control input specifies the second data input;

collectively operating the multiplexed registers of the N PUs as an N-word rotater that rotates by 2{circumflex over ( )}J words when the control input specifies the third data input; and

collectively operating the multiplexed registers of the N PUs as an N-word rotater that rotates by 2{circumflex over ( )}K words when the control input specifies the fourth data input.

12. The method of claim 11 ,

executing, by the array of N PUs, instructions of a program that 3-dimensionally convolves a W column by H row by C channel input with F of S column by R row by C channel filters to generate F of P column by Q row outputs; and

wherein said executing the program instructions comprises:

(a) S times, selecting, by the control input, the second data input and multiplying, by the arithmetic unit, the second and third inputs to generate a product and accumulating the product with the first input to generate the result; and

(b) one or more times, selecting, by the control input, the third or fourth data input to accomplish rotating by B minus S words, wherein B is the smallest factor of N that is at least as great as W.

13. The method of claim 12 ,

wherein said executing the program instructions further comprises:

(c) S times concurrent with operation (a), providing, by the first memory, the N weight words of a row to the corresponding weight inputs of the N PUs of the PU array.

14. The method of claim 13 ,

wherein said executing the program instructions further comprises:

(d) performing operations (a), (b) and (c) at least C times.

15. The method of claim 14 ,

wherein said executing the program instructions further comprises:

(e) prior to operation (d), providing, by the second memory, the N data words of a row to the corresponding first data inputs of the multiplexed register of the N PUs of the PU array.

16. The method of claim 15 ,

wherein the row provided by the second memory contains a horizontal slice of the input corresponding to one of the H rows of the input.

17. The method of claim 16 ,

wherein each of the S times C rows provided by the first memory are logically partitioned as F filter blocks, wherein each filter block of the F filter blocks contains P copies of a different weight from a corresponding one of the F filters.

18. The method of claim 17 ,

wherein a location within the one of the F filters of the different weight is determined at least in part by which of the S times operations (a) and (c) are performed and which of the at least C times operation (d) is performed.

19. The method of claim 11 ,

wherein to perform the operation on the first, second and third inputs, the arithmetic unit multiplies the second and third inputs to generate a product and accumulates the product with the first input to generate the result.

20. The method of claim 11 ,

wherein the multiplexed register of each PU of the array of N PUs further has a fifth data input, and the control input controls selection of the first, second, third, fourth and fifth data inputs;

wherein the output of the multiplexed register is also received by:

the fifth data input of the multiplexed register of a PU 2{circumflex over ( )}L PUs away, wherein L is an integer greater than K; and

wherein the multiplexed registers of the N PUs collectively operate as an N-word rotater that rotates by 2{circumflex over ( )}L words when the control input specifies the fifth data input.

21. A computer program product encoded in at least one non-transitory computer usable medium for use with a computing device, the computer program product comprising:

computer usable program code embodied in said medium, for specifying an apparatus, the computer usable program code comprising:

first program code for specifying an array of N processing units (PU) each having:

an accumulator having an output;

an arithmetic unit having first, second and third inputs and that performs an operation thereon to generate a result to store in the accumulator, the first input receives the output of the accumulator;

a weight input that is received by the second input to the arithmetic unit; and

a multiplexed register having first, second, third and fourth data inputs, an output received by the third input to the arithmetic unit, and a control input that controls selection of the first, second, third and fourth data inputs;

second program code for specifying a first memory that holds rows of N weight words and provides the N weight words of a row to the corresponding weight inputs of the N PUs of the PU array;

third program code for specifying a second memory that holds rows of N data words and provides the N data words of a row to the corresponding first data inputs of the multiplexed register of the N PUs of the PU array;

wherein the output of the multiplexed register is also received by:

the second data input of the multiplexed register of a PU one PU away;

the third data input of the multiplexed register of a PU 2{circumflex over ( )}J PUs away, wherein J is an integer greater than 1; and

the fourth data input of the multiplexed register of a PU 2{circumflex over ( )}K PUs away, wherein K is an integer greater than J;

wherein the multiplexed registers of the N PUs collectively operate as an N-word rotater that rotates by one word when the control input specifies the second data input;

wherein the multiplexed registers of the N PUs collectively operate as an N-word rotater that rotates by 2{circumflex over ( )}J words when the control input specifies the third data input; and

wherein the multiplexed registers of the N PUs collectively operate as an N-word rotater that rotates by 2{circumflex over ( )}K words when the control input specifies the fourth data input.