High throughput matrix processor with support for concurrently processing multiple matrices

1. A system, comprising:

a data input vector unit configured to concurrently receive elements of a plurality of different rows of a first data matrix and a second data matrix;

a weight input vector unit configured to:

receive a combined weight vector, wherein the combined weight vector includes weight elements of a first weight matrix and weight elements of a second weight matrix, and the first weight matrix and the second weight matrix are different matrices for different corresponding data matrices despite being combined in the combined weight vector;

extract for the first data matrix the weight elements of the first weight matrix from the combined weight vector;

extract for the second data matrix the weight elements of the second weight matrix from the combined weight vector; and

at least in part concurrently provide the extracted weight elements of the first weight matrix to a first group of one or more calculation units for the first data matrix and provide the extracted weight elements of the second weight matrix to a second group of one or more calculation units for the second data matrix; and

a plurality of calculation units of a matrix processor unit including the first group of one or more calculation units and the second group of one or more calculation units,

wherein each calculation unit of the first group of calculation units is configured to multiply provided elements of the first data matrix from the data input vector unit with provided corresponding elements of the first weight matrix from the weight input vector unit and sum together multiplication results of the corresponding calculation unit to at least in part determine a corresponding element in a first convolution result matrix, and

wherein each calculation unit of the second group of calculation units is configured to multiply provided elements of the second data matrix from the data input vector unit with provided corresponding elements of the second weight matrix from the weight input vector unit and sum together multiplication results of the corresponding calculation unit to at least in part determine a corresponding element in a second convolution result matrix.

2. The system of claim 1 , wherein each of the plurality of calculation units is a different vector calculation unit.

3. The system of claim 2 , wherein each vector calculation unit includes a vector multiply unit and a vector adder unit.

4. The system of claim 3 , wherein the vector adder unit includes an adder tree.

5. The system of claim 1 , wherein the matrix processor unit includes one or more vector accumulators.

6. The system of claim 5 , wherein at least one of the one or more vector accumulators is configured to store a vector of intermediate results corresponding to partial results of convoluting the first weight matrix with the first data matrix.

7. The system of claim 1 , wherein the first weight matrix and the second weight matrix are different machine learning weight matrices and the first and second data matrices store data for machine learning.

8. The system of claim 1 , wherein the first and second data matrices correspond to different channels of a three-dimensional machine learning data matrix.

9. The system of claim 1 , wherein the first and second weight matrices are each a 3×3 matrix.

10. The system of claim 1 , wherein the weight input vector unit includes a first register for storing extracted weight elements of the first weight matrix and a second register for storing extracted weight elements of the second weight matrix, and wherein the first register and second register are different.

11. The system of claim 10 , wherein the weight input vector unit includes a shifting circuit configured to shift each of the extracted weight elements within the first register and each of the extracted weight elements within the second register.

12. The system of claim 1 , wherein the weight input vector unit is further configured to format the extracted weight elements of the first weight matrix into a first weight input mask and to format the extracted weight elements of the second weight matrix into a second weight input mask.

13. The system of claim 12 , wherein the first weight input mask is stored in a first register of the weight input vector unit and the second weight input mask is stored in a second register of the weight input vector unit, and wherein the first register and second register are different.

14. The system of claim 12 , wherein each of the first and second weight input masks includes zero-value padding elements.

15. The system of claim 1 , wherein the combined weight vector is a 32-byte vector stored in a memory unit and includes zero-value padding elements.

16. The system of claim 1 , wherein the combined weight vector includes weight elements of three or more different weight matrices.

17. A method, comprising:

receiving at a processing element an instruction specifying a first data matrix, a second data matrix, and a combined weight vector, wherein the first data matrix is associated with a first convolution operation and the second data matrix is associated with a second convolution operation, and wherein the combined weight vector includes weight elements of a first weight matrix associated with the first convolution operation and weight elements of a second weight matrix associated with the second convolution operation, and the first weight matrix and the second weight matrix are different matrices for different corresponding data matrices despite being combined in the combined weight vector;

extracting for the first data matrix the weight elements of the first weight matrix from the combined weight vector;

extracting for the second data matrix the weight elements of the second weight matrix from the combined weight vector;

assigning a plurality of calculation units of a matrix processor unit to a first group of one or more calculation units and a different plurality of calculation units to a second group of one or more calculation units;

creating a first weight input mask associated with the first convolution operation from the combined weight vector; and

creating a second weight input mask associated with the second convolution operation from the combined weight vector.

18. The method of claim 17 , further comprising:

broadcasting the first weight input mask to each calculation unit of the first group of one or more calculation units;

broadcasting the first weight input mask to each calculation unit of the second group of one or more calculation units;

providing each calculation unit of the first group of one or more calculation units elements of a plurality of different rows of the first data matrix;

providing each calculation unit of the second group of one or more calculation units elements of a plurality of different rows of the second data matrix; and

at least in part concurrently performing vector operations using the first and second groups of one or more calculation units using a broadcasted first or second weight input and provided data elements from the first or second data matrix.

19. A method, comprising:

concurrently receiving at a data input vector unit elements of a plurality of different rows of a first data matrix and a second data matrix;

receiving at a weight input vector unit a combined weight vector, wherein the combined weight vector includes weight elements of a first weight matrix and weight elements of a second weight matrix, and the first weight matrix and the second weight matrix are different matrices for different corresponding data matrices despite being combined in the combined weight vector;

extracting for the first data matrix the weight elements of the first weight matrix from the combined weight vector;

extracting for the second data matrix the weight elements of the second weight matrix from the combined weight vector;

at least in part concurrently providing the extracted weight elements of the first weight matrix to a first group of one or more calculation units and providing the extracted weight elements of the second weight matrix to a second group of one or more calculation units; and

at least in part concurrently providing the elements of the plurality of the different rows of the first data matrix first to the first group of the one or more calculation units for the first data matrix and providing the elements of the plurality of the different rows of the second data matrix first to the second group of the one or more calculation units for the second data matrix.

20. The method of claim 19 , further comprising:

at least in part concurrently multiplying the provided elements of the first data matrix from the data input vector unit with the provided corresponding elements of the first weight matrix from the weight input vector unit and multiplying the provided elements of the second data matrix from the data input vector unit with the provided corresponding elements of the second weight matrix from the weight input vector unit; and

at least in part concurrently summing together multiplication results to at least in part determine a corresponding element in a first convolution result matrix and summing together multiplication results to at least in part determine a corresponding element in a second convolution result matrix.