Neural network unit with neural memory and array of neural processing units that collectively shift row of data received from neural memory

1. An apparatus, comprising: an array of N processing units (PU) each having: an accumulator having an output; an arithmetic unit having first, second and third inputs and that performs an operation thereon to generate a result to store in the accumulator, the first input receives the output of the accumulator; a weight input that is received by the second input to the arithmetic unit; and a multiplexed register having first and second data inputs, an output received by the third input to the arithmetic unit, and a control input that controls selection of the first and second data inputs; and the output of the multiplexed register is also received by the second data input of the multiplexed register of an adjacent PU, the multiplexed registers of the N PUs collectively operate as an N-word rotater when the control input specifies the second data input; a first memory that holds W rows of N weight words and provides the N weight words of a row of the W rows to the corresponding weight inputs of the N PUs of the PU array; and a second memory that holds D rows of N data words and provides the N data words of a row of the D rows to the corresponding first data inputs of the multiplexed register of the N PUs of the PU array.

2. The apparatus of claim 1 , further comprising: to perform the operation, the arithmetic unit multiplies the second and third inputs to generate a product and accumulates the product with the first input to generate the result.

3. The apparatus of claim 1 , further comprising: to perform the operation, the arithmetic unit generates a difference between the second and third inputs and accumulates the difference with the first input to generate the result.

4. The apparatus of claim 1 , further comprising: the apparatus is comprised by a processor having an instruction set and an architectural instruction thereof that instructs the processor to write a plurality of data words from one or more architectural registers of the processor to a location in the second memory specified by the architectural instruction.

5. The apparatus of claim 4 , further comprising: the second memory includes: a first port that provides the N data words of the row of the D rows to the N PUs; and a second port that receives the plurality of data words for writing to the location in the second memory concurrently with the first port providing the N data words to the N PUs.

6. The apparatus of claim 1 , further comprising: the apparatus is comprised by a processor having an instruction set and an architectural instruction thereof that instructs the processor to write a plurality of weight words from one or more architectural registers of the processor to a location in the first memory specified by the architectural instruction.

7. The apparatus of claim 6 , further comprising: a buffer that buffers N weight words for writing to a row of the W rows of the first memory after being populated by execution of a plurality of instances of the architectural instruction.

8. The apparatus of claim 1 , further comprising: the apparatus is comprised by a processor having an instruction set and an architectural instruction thereof that instructs the processor to read a plurality of data words into one or more architectural registers of the processor from a location in the second memory specified by the architectural instruction.

9. The apparatus of claim 1 , further comprising: W is at least N.

10. The apparatus of claim 1 , further comprising: N is at least 512.

11. A processor, comprising: an instruction set having architectural instructions that instruct the processor to perform actions; and an array of N processing units (PU) each having: an accumulator having an output; an arithmetic unit having first, second and third inputs and that performs an operation thereon to generate a result to store in the accumulator, the first input receives the output of the accumulator; a weight input that is received by the second input to the arithmetic unit; and a multiplexed register having first and second data inputs, an output received by the third input to the arithmetic unit, and a control input that controls selection of the first and second data inputs; and the output of the multiplexed register is also received by the second data input of the multiplexed register of an adjacent PU, the multiplexed registers of the N PUs collectively operate as an N-word rotater when the control input specifies the second data input; a first memory that holds W rows of N weight words and provides the N weight words of a row of the W rows to the corresponding weight inputs of the N PUs of the PU array; and a second memory that holds D rows of N data words and provides the N data words of a row of the D rows to the corresponding first data inputs of the multiplexed register of the N PUs of the PU array.

12. The processor of claim 11 , further comprising: to perform the operation, the arithmetic unit multiplies the second and third inputs to generate a product and accumulates the product with the first input to generate the result.

13. The processor of claim 11 , further comprising: to perform the operation, the arithmetic unit generates a difference between the second and third inputs and accumulates the difference with the first input to generate the result.

14. The processor of claim 11 , further comprising: an architectural instruction instructs the processor to write a plurality of data words from one or more architectural registers of the processor to a location in the second memory specified by the architectural instruction.

15. The processor of claim 14 , further comprising: the second memory includes: a first port that provides the N data words of the row of the D rows to the N PUs; and a second port that receives the plurality of data words for writing to the location in the second memory concurrently with the first port providing the N data words to the N PUs.

16. The processor of claim 11 , further comprising: an architectural instruction instructs the processor to write a plurality of weight words from one or more architectural registers of the processor to a location in the first memory specified by the architectural instruction.

17. The processor of claim 16 , further comprising: a buffer that buffers N weight words for writing to a row of the W rows of the first memory after being populated by execution of a plurality of instances of the architectural instruction.

18. The processor of claim 11 , further comprising: an architectural instruction instructs the processor to read a plurality of data words into one or more architectural registers of the processor from a location in the second memory specified by the architectural instruction.

19. The processor of claim 11 , further comprising: W is at least N.

20. The processor of claim 11 , further comprising: N is at least 512.

21. A computer program product encoded in at least one non-transitory computer usable medium for use with a computing device, the computer program product comprising: computer usable program code embodied in said medium, for specifying an apparatus, the computer usable program code comprising: first program code for specifying an array of N processing units (PU) each having: an accumulator having an output; an arithmetic unit having first, second and third inputs and that performs an operation thereon to generate a result to store in the accumulator, the first input receives the output of the accumulator; a weight input that is received by the second input to the arithmetic unit; and a multiplexed register having first and second data inputs, an output received by the third input to the arithmetic unit, and a control input that controls selection of the first and second data inputs; and the output of the multiplexed register is also received by the second data input of the multiplexed register of an adjacent PU, the multiplexed registers of the N PUs collectively operate as an N-word rotater when the control input specifies the second data input; second program code for specifying a first memory that holds W rows of N weight words and provides the N weight words of a row of the W rows to the corresponding weight inputs of the N PUs of the PU array; and third program code for specifying a second memory that holds D rows of N data words and provides the N data words of a row of the D rows to the corresponding first data inputs of the multiplexed register of the N PUs of the PU array.