Mapping convolution to a matrix processor unit
A system comprises a matrix processor unit that includes a first type of register, a group of a second type of registers, and a plurality of calculation units. The first type of register is configured to concurrently store values from different rows of a first matrix. At least a portion of the first type of register is logically divided into groups of elements, and each of the groups corresponds to a different row of the first matrix. Each of the second type of registers is configured to concurrently store values from a plurality of different rows of a second matrix. Each of the calculation units corresponds to one of the second type of registers and is configured to at least in part determine a corresponding element in a result matrix of convoluting the second matrix with the first matrix.
1. A system, comprising:
a matrix processor unit, including:
a first register configured to concurrently store values of a plurality of different rows of a first matrix, wherein at least a portion of the first register is logically divided into groups of elements, and each group of the groups of elements corresponds to a different row of the first matrix;
a group of registers, wherein each register of the group of registers is configured to concurrently store values from a plurality of different rows of a second matrix, and each register of the group of registers is configured to store zero-value padding elements; and
a plurality of calculation units, wherein each calculation unit of the plurality of calculation units corresponds to a corresponding one of the group of registers, and each calculation unit of the plurality of calculation units is configured to multiply each value stored in the first register with a corresponding value stored in the corresponding one of the group of registers and sum together multiplication results of the corresponding calculation unit to at least in part determine a corresponding element in a result matrix of convoluting the second matrix with the first matrix.
2. The system of claim 1 , wherein the first matrix is a machine learning weight matrix and the second matrix is a machine learning data matrix.
3. The system of claim 1 , wherein each element of the groups of elements of the first register is separated by zero-value elements.
4. The system of claim 1 , wherein each calculation unit of the plurality of calculation units includes a vector multiply unit and a vector adder unit.
5. The system of claim 4 , wherein the vector adder unit includes an adder tree.
6. The system of claim 1 , wherein the matrix processor unit includes a vector accumulator.
7. The system of claim 6 , wherein the vector accumulator is configured to store a vector of intermediate results corresponding to partial results of convoluting the second matrix with the first matrix.
8. The system of claim 1 , wherein the first register is configured to broadcast contents to each calculation unit of the plurality of calculation units.
9. The system of claim 1 , wherein the first register is a flip-flop circuit.
10. The system of claim 1 , wherein each value stored in the first register is a 4-bit, 8-bit, 2-byte, or 4-byte element.
11. The system of claim 1 , wherein the first register includes a shifting circuit configured to shift each group of the groups of elements within the first type of register.
12. The system of claim 1 , further comprising a communication bus; a data bus; and a control logic unit.
13. The system of claim 12 , wherein the control logic unit is configured to receive a convolution operation instruction specifying the first matrix and the second matrix via the communication bus.
14. The system of claim 12 , wherein the data bus is configured to read values of the second matrix at a multiple of a clock speed of the matrix processor unit.
15. The system of claim 1 , wherein the first matrix is a 3×3, 5×5, 7×7, 9×9, or 11×11 matrix.
16. The system of claim 1 , wherein the result matrix of convoluting the second matrix with the first matrix includes depthwise convolution results.
17. A method, comprising:
receiving a depthwise convolution operation instruction specifying a first matrix and a second matrix;
preparing in a first register a weight input mask using elements of the first matrix;
broadcasting the weight input mask to a plurality of vector calculation units of a matrix processor unit; and
for each vector calculation unit of the plurality of vector calculation units:
identifying a two-dimensional slice of the second matrix;
storing identified elements of the slice in a second register, wherein the second register stores zero-value padding elements;
loading stored contents of the second register into the vector calculation unit as a corresponding data input vector;
multiplying each value of the weight input mask with a corresponding value in the corresponding data input vector to determine multiplication results; and
summing together the multiplication results of the corresponding vector calculation unit to at least in part determine a corresponding element in a result matrix of convoluting the second matrix with the first matrix.
18. The method of claim 17 , further comprising:
shifting the elements of the weight input mask to align elements of the weight input mask with a different set of elements of the second matrix;
broadcasting the shifted weight input mask to the plurality of vector calculation units of the matrix processor unit; and
for each vector calculation unit of the plurality of vector calculation units:
multiplying each value of the shifted weight input mask with a corresponding value in the corresponding data input vector to determine multiplication results; and
summing together the multiplication results of the corresponding vector calculation unit to at least in part determine a second corresponding element in the result matrix of convoluting the second matrix with the first matrix.
19. The method of claim 17 , wherein the weight input mask includes only a subset of rows of the first matrix.
20. A system, comprising:
a shared memory; and
a plurality of processing elements configured to operate in parallel, wherein each processing element includes:
a matrix processor unit, including:
a first register configured to concurrently store values of a plurality of different rows of a first matrix, wherein at least a portion of the first register is logically divided into groups of elements, and each group of the groups of elements corresponds to a different row of the first matrix;
a group of registers, wherein each register of the group of registers is configured to concurrently store values from a plurality of different rows of a second matrix, and each register of the group of registers is configured to store zero-value padding elements; and
a plurality of calculation units, wherein each calculation unit of the plurality of calculation units corresponds to a corresponding one of the group of registers, and each calculation unit of the plurality of calculation units is configured to multiply each value stored in the first register with a corresponding value stored in the corresponding one of the group of registers and sum together multiplication results of the corresponding calculation unit to at least in part determine a corresponding element in a result matrix of convoluting the second matrix with the first matrix.