Low latency matrix multiply unit

1. A cell of a plurality of cells arranged in an array of a matrix multiply unit, the cell comprising:

weight shift registers configured to receive, over two shift chains, weight values of a neural network, wherein each weight shift register of the weight shift registers is connected to only one shift chain of the two shift chains;

a weight matrix register coupled to the weight shift registers, the weight matrix register configured to store the weight values received from the weight shift registers; and

a multiply unit coupled to the weight matrix register, the multiply unit configured to receive at least one weight value of the weight values from the weight matrix register, the multiply unit configured to receive a data input from a vector register, the multiply unit configured to multiply the at least one weight value with the data input to determine a multiplication result.

2. The cell of claim 1 , wherein the array is a two-dimensional array.

3. The cell of claim 1 , wherein the plurality of cells are arranged in a plurality of columns of the array.

4. The cell of claim 1 , wherein:

the weight values are transmitted in pairs over the two shift chains from the vector register to the weight shift registers when the weight values are available from the vector register; and

the weight values are received from a holding register when the weight values are unavailable from the vector register.

5. The cell of claim 4 , wherein:

when the weight values are available from the vector register, at least one pair of the weight values is shifted on a clock cycle to the weight shift registers.

6. The cell of claim 4 , wherein

when the weight values are unavailable from the vector register:

on a first clock cycle when a first weight value of the weight values becomes available, the holding register is loaded with the first weight value as a held value that is not transmitted over the two shift chains at the first clock cycle; and

on a subsequent clock cycle when a second weight value of the weight values becomes available, the second weight value and the held value are transmitted over the two shift chains to the weight shift registers, each shift chain of the two shift chains transmitting a corresponding held value.

7. The cell of claim 4 , wherein the vector register is configured to store the weight values as packed sets of four 8-bit integers.

8. The cell of claim 7 , wherein:

each shift chain of the two shift chains has two injection points at different locations on the shift chain for injecting the weight values, the two injection points being a first injection point and a second injection point;

two of the four 8-bit integers are injected at the first injection point; and

the other two of the four 8-bit integers are injected at the second injection point.

9. The cell of claim 1 , wherein:

when the weight values are in the weight matrix register, the weight values are used by the multiply unit to compute additional multiplication results; and

during the computation of the additional multiplication results, additional weight values are transmitted into the weight shift registers in preparation for computing further multiplication results after the additional multiplication results.

10. A method performed by a cell of a plurality of cells arranged in an array, the method comprising:

receiving, by weight shift registers within the cell, weight values of a neural network over two shift chains;

transmitting, by the weight shift registers and to a weight matrix register within the cell, the weight values;

storing, by the weight matrix register, the weight values;

transmitting, by the weight matrix register, at least one weight value of the weight values to a multiply unit within the cell;

receiving, by the multiply unit, a data input from a vector register; and

multiplying, by a multiply unit coupled to the weight matrix register, the at least one weight value with the data input.

11. The method of claim 10 , wherein the array is a two-dimensional array.

12. The method of claim 10 , wherein the plurality of cells are arranged in a plurality of columns of the array.

13. The method of claim 10 , wherein the receiving of the weight values by the weight shift registers comprises:

receiving the weight values in pairs over the two shift chains from the vector register when the weight values are available from the vector register; and

receiving the weight values from a holding register when the weight values are unavailable from the vector register.

14. The method of claim 13 , further comprising:

shifting, when the weight values are available from the vector register, at least one pair of the weight values on a clock cycle to the weight shift registers.

15. The method of claim 13 , further comprising:

when the weight values are unavailable from the vector register:

loading, on a first clock cycle when a first weight value of the weight values becomes available, the holding register with the first weight value as a held value that is not transmitted over the two shift chains at the first clock cycle; and

transmitting, on a subsequent clock cycle when a second weight value of the weight values becomes available, the second weight value and the held value over the two shift chains to the weight shift registers, each shift chain of the two shift chains transmitting a corresponding held value.

16. The method of claim 13 , wherein:

the vector register is configured to store the weight values as packed sets of four 8-bit integers; and

each shift chain of the two shift chains has two injection points at different locations on the shift chain for injecting the weight values, the two injection points being a first injection point and a second injection point.

17. The method of claim 16 , further comprising:

injecting two of the four 8-bit integers at the first injection point; and

injecting the other two of the four 8-bit integers at the second injection point.

18. The method of claim 10 , further comprising:

computing, by the multiply unit when the weight values are in the weight matrix register, additional multiplication results by using the weight values; and

transmitting, during the computation of the additional multiplication results, additional weight values into the weight shift registers in preparation for computing by the multiply unit further multiplication results after the additional multiplication results.