Sparsity-aware compute-in-memory
Certain aspects of the present disclosure provide techniques for performing machine learning computations in a compute in memory (CIM) array comprising a plurality of bit cells, including: determining that a sparsity of input data to a machine learning model exceeds an input data sparsity threshold; disabling one or more bit cells in the CIM array based on the sparsity of the input data prior to processing the input data; processing the input data with bit cells not disabled in the CIM array to generate an output value; applying a compensation to the output value based on the sparsity to generate a compensated output value; and outputting the compensated output value.
1. A method, comprising:
disabling one or more bit cells in a compute-in-memory (CIM) array based on a sparsity of input data for a machine learning model prior to processing the input data;
determining that a sparsity of weight data for the machine learning model exceeds a weight data sparsity threshold;
resequencing the weight data according to the sparsity of the weight data;
disabling one or more bit cells in the CIM array based on the resequenced weight data;
resequencing the input data based on the resequenced weight data;
processing the input data with bit cells not disabled in the CIM array to generate an output value;
applying a compensation to the output value based on the sparsity of the input data to generate a compensated output value; and
outputting the compensated output value.
2. The method of claim 1 , wherein disabling one or more bit cells in the CIM array based on the sparsity of the input data prior to processing the input data comprises disabling each row of the CIM array for which an associated activation input value is 0.
3. The method of claim 1 , wherein the compensation is applied in a digital domain.
4. The method of claim 1 , further comprising:
determining whether the input data comprises binary input data or multibit input data;
setting bit cells of the CIM array to an AND operation mode if the input data comprises multibit input data; and
setting bit cells of the CIM array to an XNOR operation mode if the input data comprises binary input data.
5. The method of claim 1 , wherein resequencing the weight data according to the sparsity of the weight data comprises:
resequencing rows of a CIM array weight matrix according to a sparsity of each row in the CIM array weight matrix; and
resequencing columns of the CIM array weight matrix according to a sparsity of each column in the CIM array weight matrix.
6. The method of claim 5 , wherein disabling one or more bit cells in the CIM array based on the resequenced weight data comprises disabling one or more tiles of bit cells, wherein each tile of the one or more tiles comprises a plurality of bit cells with weight values of 0.
7. The method of claim 1 , further comprising:
compressing the resequenced weight data by removing any weight with a value of 0; and
storing the compressed resequenced weight data in a memory.
8. The method of claim 7 , further comprising:
retrieving the compressed resequenced weight data from the memory;
decompressing the compressed resequenced weight data; and
writing the decompressed resequenced weight data to the CIM array prior to processing the input data with the bit cells not disabled in the CIM array to generate the output value.
9. The method of claim 1 , wherein determining the sparsity of the input data comprises determining a count of activation inputs in the input data with an activation input value of 0.
10. The method of claim 1 , further comprising determining that the sparsity of the input data to the machine learning model exceeds an input data sparsity threshold.
11. A method, comprising:
resequencing weight data for a machine learning model according to a sparsity of the weight data, wherein the sparsity of the weight data exceeds a weight data sparsity threshold;
disabling one or more bit cells in a compute-in-memory (CIM) array based on the resequenced weight data;
resequencing input data based on the resequenced weight data;
processing the input data with bit cells not disabled in the CIM array to generate an output value;
applying a compensation to the output value based on the sparsity to generate a compensated output value; and
outputting the compensated output value.
12. The method of claim 11 , wherein:
disabling one or more bit cells in the CIM array based on the resequenced weight data comprises disabling one or more tiles of bit cells, and
each tile of the one or more tiles comprises a plurality of bit cells with weight values of 0.
13. The method of claim 11 , further comprising:
compressing the resequenced weight data by removing any weight with a value of 0; and
storing the compressed resequenced weight data in a memory.
14. The method of claim 13 , further comprising:
retrieving the compressed resequenced weight data from the memory;
decompressing the compressed resequenced weight data; and
writing the decompressed resequenced weight data to the CIM array prior to processing the input data with bit cells not disabled in the CIM array to generate the output value.
15. A processing system, comprising:
a memory comprising computer-executable instructions; and
one or more processors configured to execute the computer-executable instructions and cause the processing system to:
disable one or more bit cells in a compute-in-memory (CIM) array based on a sparsity of input data for a machine learning model prior to processing the input data;
determine that a sparsity of weight data for the machine learning model exceeds a weight data sparsity threshold;
resequence the weight data according to the sparsity of the weight data;
disable one or more bit cells in the CIM array based on the resequenced weight data;
resequence the input data based on the resequenced weight data;
process the input data with bit cells not disabled in the CIM array to generate an output value;
apply a compensation to the output value based on the sparsity of the input data to generate a compensated output value; and
output the compensated output value.
16. The processing system of claim 15 , wherein in order to disable one or more bit cells in the CIM array based on the sparsity of the input data prior to processing the input data, the one or more processors are further configured to disable each word line of the CIM array for which an associated activation input value is 0.
17. The processing system of claim 15 , wherein the compensation is applied in a digital domain.
18. The processing system of claim 15 , wherein the one or more processors are further configured to:
determine whether the input data comprises binary input data or multibit input data;
set bit cells of the CIM array to an AND operation mode if the input data comprises multibit input data; and
set bit cells of the CIM array to an XNOR operation mode if the input data comprises binary input data.
19. The processing system of claim 15 , wherein in order to resequence the weight data according to the sparsity of the weight data, the one or more processors are further configured to:
resequence rows of a CIM array weight matrix according to a sparsity of each row in the CIM array weight matrix; and
resequence columns of the CIM array weight matrix according to a sparsity of each column in the CIM array weight matrix.
20. The processing system of claim 19 , wherein in order to disable one or more bit cells in the CIM array based on the resequenced weight data, the one or more processors are further configured to disable one or more tiles of bit cells, wherein each tile of the one or more tiles comprises a plurality of bit cells with weight values of 0.
21. The processing system of claim 15 , wherein the one or more processors are further configured to:
compress the resequenced weight data by removing any weight with a value of 0; and
store the compressed resequenced weight data in a memory.
22. The processing system of claim 21 , wherein the one or more processors are further configured to:
retrieve the compressed resequenced weight data from the memory;
decompress the compressed resequenced weight data; and
write the decompressed resequenced weight data to the CIM array prior to processing the input data with the bit cells not disabled in the CIM array to generate the output value.
23. The processing system of claim 15 , wherein in order to determine the sparsity of the input data, the one or more processors are further configured to determine a count of activation inputs in the input data with an activation input value of 0.
24. The processing system of claim 15 , wherein the one or more processors are further configured to determine that the sparsity of the input data to the machine learning model exceeds an input data sparsity threshold.
25. A processing system, comprising:
a memory comprising computer-executable instructions; and
one or more processors configured to execute the computer-executable instructions and cause the processing system to:
resequence weight data for a machine learning model according to a sparsity of the weight data, wherein the sparsity of the weight data exceeds a weight data sparsity threshold;
disable one or more bit cells in a compute-in-memory (CIM) array based on the resequenced weight data;
resequence input data based on the resequenced weight data;
process the input data with bit cells not disabled in the CIM array to generate an output value;
apply a compensation to the output value based on the sparsity to generate a compensated output value; and
output the compensated output value.
26. The processing system of claim 25 , wherein:
in order to disable one or more bit cells in the CIM array based on the resequenced weight data, the one or more processors are further configured to disable one or more tiles of bit cells, and
each tile of the one or more tiles comprises a plurality of bit cells with weight values of 0.
27. The processing system of claim 25 , wherein the one or more processors are further configured to:
compress the resequenced weight data by removing any weight with a value of 0; and
store the compressed resequenced weight data in a memory.
28. The processing system of claim 27 , wherein the one or more processors are further configured to:
retrieve the compressed resequenced weight data from the memory;
decompress the compressed resequenced weight data; and
write the decompressed resequenced weight data to the CIM array prior to processing the input data with bit cells not disabled in the CIM array to generate the output value.