In situ sparse matrix expansion
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for loading a matrix into a circuit having an array having M×N cells. One of the methods includes: receiving a plurality of non-zero input values from a first input matrix; receiving index metadata that indicates, for each non-zero input value in the plurality of input values, which cell of the M×N cells in the array the non-zero input value should be loaded into; sending the non-zero input values and the index metadata to the M×N cells; and at a particular cell of the M×N cells in the array: receiving a particular non-zero input value and corresponding index metadata; and determining from the corresponding index metadata for the particular non-zero input value whether to store the particular non-zero input value at the cell or to shift the particular non-zero input value to another cell.
1. A method performed by a hardware device comprising circuitry for an array having M×N cells, the method comprising:
receiving, at the hardware device, a plurality of non-zero input values from a first input matrix to be multiplied with a second input matrix using the array, the first input matrix comprising the plurality of non-zero input values and a plurality of zero input values;
receiving, at the hardware device, index metadata that indicates, for each non-zero input value in the plurality of non-zero input values, which cell of the M×N cells in the array a non-zero input value should be loaded into;
sending, using the hardware device, the plurality of non-zero input values and the index metadata to the M×N cells, wherein the sending comprises,
at a particular cell of the M×N cells in the array:
receiving a particular non-zero input value and corresponding index metadata for the particular non-zero input value; and
determining from the corresponding index metadata for the particular non-zero input value whether to store the particular non-zero input value at the particular cell for use in performing multiplication or to shift the particular non-zero input value to another cell,
wherein M and N are positive integers larger than one, and wherein, after the sending, one or more cells of the M×N cells do not store a non-zero input value; and
performing the multiplication by operating each of the one or more cells that do not store a non-zero input value as if a zero input value had been loaded into each of the one or more cells.
2. The method of claim 1 , further comprising, at the particular cell of the M×N cells in the array and prior to sending the plurality of non-zero input values and the index metadata to the M×N cells: automatically loading a zero input value into the particular cell.
3. The method of claim 1 , wherein sending the plurality of non-zero input values and the index metadata to the MXN cells comprises sending the plurality of non-zero input values along a first dimension of the array that has the M×N cells.
4. The method of claim 3 , wherein the first dimension of the array comprises a vertical dimension of the array; and
sending the plurality of non-zero input values and the index metadata to the M×N cells along the first dimension of the array comprises sending the plurality of non-zero input values and the index metadata beginning from a top fringe of the array.
5. The method of claim 3 , wherein the first dimension of the array comprises a horizontal dimension of the array; and
sending the plurality of non-zero input values and the index metadata to the M×N cells along the first dimension of the array comprises sending the plurality of non-zero input values and the index metadata beginning from a left fringe of the array.
6. The method of claim 1 , wherein the index metadata comprises an absolute fixed-length row index for each non-zero input value.
7. The method of claim 1 , wherein the index metadata comprises a run-length encoded row index for each non-zero input value.
8. The method of claim 1 , wherein the index metadata comprises a bitmap of locations of the plurality of non-zero input values in the M×N cells of the array.
9. The method of claim 3 , wherein the hardware device further comprises a comparator for each cell in the M×N cells of the array, the comparator configured to compare (i) the index metadata for each non-zero input value with (ii) an index of a respective cell that indicates its location in the M×N cells along the first dimension of the array.
10. The method of claim 1 , wherein the plurality of non-zero input values are in a compressed sparse column (CSC) matrix format.
11. The method of claim 1 , wherein the hardware device further comprises a first-in, first-out (FIFO) register for each cell in the M×N cells of the array, and wherein loading a respective non-zero input value into each cell comprises enqueuing the respective non-zero input value in the FIFO register associated with each cell.
12. The method of claim 1 , further comprising, for each cell in the array storing a particular non-zero input value:
receiving, at the hardware device, a second input value from the second input matrix; and
determining, using the hardware device, a respective multiplication product based on the particular non-zero input value and the second input value.
13. The method of claim 1 , wherein the plurality of non-zero input values move by one cell per clock cycle.
14. The method of claim 1 , further comprising:
determining that the first input matrix includes a column that only has zero input values; and
in response, adding a sign bit to an index defined by corresponding index metadata for each zero input value in the column.
15. A system comprising one or more computers and computer-readable medium coupled to the one or more computers and having instructions stored thereon, which, when executed by the one or more computers, cause the one or more computers to perform operations comprising:
receiving, at a hardware device comprising circuitry for an array having M×N cells, a plurality of non-zero input values from a first input matrix to be multiplied with a second input matrix using the array, the first input matrix comprising the plurality of non-zero input values and a plurality of zero input values;
receiving, at the hardware device, index metadata that indicates, for each non-zero input value in the plurality of non-zero input values, which cell of the M×N cells in the array a non-zero input value should be loaded into;
sending, using the hardware device, the plurality of non-zero input values and the index metadata to the MXN cells, wherein the sending comprises,
at a particular cell of the M×N cells in the array:
receiving a particular non-zero input value and corresponding index metadata for the particular non-zero input value; and
determining from the corresponding index metadata for the particular non-zero input value whether to store the particular non-zero input value at the particular cell for use in performing multiplication or to shift the particular non-zero input value to another cell,
wherein M and N are positive integers larger than one, and wherein, after the sending, one or more cells of the M×N cells do not store a non-zero input value; and
performing the multiplication by operating each of the one or more cells that do not store a non-zero input value as if a zero input value had been loaded into each of the one or more cells.
16. The system of claim 15 , wherein the operations further comprise, at the particular cell of the M×N cells in the array and prior to sending the plurality of non-zero input values and the index metadata to the M×N cells: automatically loading a zero input value into the particular cell.
17. The system of claim 15 , wherein sending the plurality of non-zero input values and the index metadata to the M×N cells comprises sending the plurality of non-zero input values along a first dimension of the array that has the M×N cells.
18. The system of claim 17 , wherein the first dimension of the array comprises a vertical dimension of the array; and
sending the plurality of non-zero input values and the index metadata to the M×N cells along the first dimension of the array comprises sending the plurality of non-zero input values and the index metadata beginning from a top fringe of the array.
19. The system of claim 17 , wherein the first dimension of the array comprises a horizontal dimension of the array; and
sending the plurality of non-zero input values and the index metadata to the M×N cells along the first dimension of the array comprises sending the plurality of non-zero input values and the index metadata beginning from a left fringe of the array.
20. The system of claim 17 , wherein the hardware device further comprises a comparator for each cell in the M×N cells of the array, the comparator configured to compare (i) the index metadata for each non-zero input value with (ii) an index of a respective cell that indicates its location in the M×N cells along the first dimension of the array.
21. A non-transitory computer-readable medium having instructions stored thereon, which, when executed by one or more computers, cause the one or more computers to perform operations comprising:
receiving, at a hardware device comprising circuitry for an array having M×N cells, a plurality of non-zero input values from a first input matrix to be multiplied with a second input matrix using the array, the first input matrix comprising the plurality of non-zero input values and a plurality of zero input values;
receiving, at the hardware device, index metadata that indicates, for each non-zero input value in the plurality of non-zero input values, which cell of the M×N cells in the array a non-zero input value should be loaded into;
sending, using the hardware device, the plurality of non-zero input values and the index metadata to the M×N cells, wherein the sending comprises,
at a particular cell of the M×N cells in the array:
receiving a particular non-zero input value and corresponding index metadata for the particular non-zero input value; and
determining from the corresponding index metadata for the particular non-zero input value whether to store the particular non-zero input value at the particular cell for use in performing multiplication or to shift the particular non-zero input value to another cell,
wherein M and N are positive integers larger than one, and wherein, after the sending, one or more cells of the M×N cells do not store a non-zero input value; and
performing the multiplication by operating each of the one or more cells that do not store a non-zero input value as if a zero input value had been loaded into each of the one or more cells.