Systems and methods of compensating degradation in analog compute-in-memory (ACIM) modules
Certain aspects of the present disclosure provide techniques for performing compute in memory (CIM) computations. A device comprises a CIM module configured to apply analog weights to input data using multiply-accumulate operations to generate an output. The device further comprises a digital weight storage unit configured to store digital weight references, wherein a digital weight reference corresponds to an analog weight of the analog weights. The device also comprises a device controller configured to program the analog weights to the CIM module, cause the CIM module to process the input data, and reprogram one or more analog weights that are degraded. The digital weight references in the digital weight storage unit are populated with values from a host processing device. Degraded analog weights in the CIM module are reprogrammed based on the corresponding digital weight references from the digital weight storage unit without reference to the host processing device.
1. A device for performing compute in memory (CIM) computations, the device comprising:
a compute in memory (CIM) module configured to apply a plurality of analog weights to input data using at least one multiply-accumulate operation to generate an output;
a digital weight storage unit configured to store a plurality of digital weight references, wherein a digital weight reference of the plurality of digital weight references corresponds to an analog weight of the plurality of analog weights; and
a device controller configured to:
program the plurality of analog weights to the CIM module based on the plurality of digital weight references, wherein programing the plurality of analog weights to the CIM module comprises converting the digital weight references in the digital weight storage unit to analog values for storage in the CIM module, wherein the digital weight storage unit is further configured to send the analog values to the CIM module for storage;
cause the CIM module to process the input data; and
reprogram one or more analog weights of the plurality of analog weights that are degraded based on corresponding digital weight references stored in the digital weight storage unit,
wherein:
the plurality of digital weight references in the digital weight storage unit are populated with values from a host processing device,
degraded analog weights in the CIM module are reprogrammed based on the corresponding digital weight references from the digital weight storage unit without further reference to the host processing device, and
the digital weight storage unit comprises an error correction component configured to perform the reprogramming by detecting and correcting an error in the digital weight references stored in the digital weight storage unit, wherein detecting the error is based on detection of a check bit changing state for a corresponding digital weight reference and correcting the error is based on nulling the corresponding digital weight reference.
2. The device of claim 1 , wherein the digital weight storage unit is further configured to store no more than a configured number of most significant bits (MSBs) of one or more of the plurality of digital weight references, the configured number being smaller than a total number of bits for the one or more of the plurality of digital weight references.
3. The device of claim 1 , wherein the digital weight storage unit is further configured to store digital weight references for a selected subset of the plurality of analog weights.
4. The device of claim 3 , wherein the selected subset of the plurality of analog weights is predetermined by the host processing device in data communication with the device.
5. The device of claim 3 , wherein the selected subset of the plurality of analog weights is selected based on a characteristic of each analog weight of the plurality of analog weights.
6. The device of claim 5 , wherein the characteristic comprises an absolute value of each analog weight of the plurality of analog weights.
7. The device of claim 5 , wherein the characteristic comprises a layer of a machine learning model associated with each analog weight of the plurality of analog weights.
8. The device of claim 3 , wherein the selected subset of the plurality of analog weights is selected based on a determination that each selected analog weight in the selected subset of the plurality of analog weights exceeds a threshold impact on an accuracy of the output.
9. The device of claim 3 , wherein the selected subset of the plurality of analog weights is randomly sampled from the plurality of analog weights.
10. The device of claim 3 , wherein the selected subset of the plurality of analog weights is selected based on an association with one or more particular bit lines or word lines in the CIM module.
11. The device of claim 10 , wherein the one or more particular bit lines or word lines in the CIM module is identified based on the output having a higher sensitivity to the one or more particular bit lines or word lines than other bit lines or word lines.
12. The device of claim 3 , wherein one or more analog weights of the selected subset of the plurality of analog weights is selected based on being used in one or more shared layers of a multi-task machine learning model.
13. The device of claim 1 , wherein the digital weight storage unit is further configured to store a digital weight reference for each of the plurality of analog weights.
14. The device of claim 1 , wherein the digital weight storage unit comprises an error correction component configured to detect and correct an error in the digital weight references stored in the digital weight storage unit.
15. The device of claim 1 , wherein the device comprises one of a Storage Class Memory device, a USB drive, or a memory module.
16. A method for performing compute in memory (CIM) computations, comprising:
storing, via a digital weight storage unit, a plurality of digital weight references, wherein a digital weight reference of the plurality of digital weight references corresponds to an analog weight of a plurality of analog weights applied by a CIM module, wherein the digital weight storage unit is coupled to the CIM module and remote from a host system in which the device is installed;
programming the plurality of analog weights to the CIM module based on the plurality of digital weight references via a device controller, wherein programing the plurality of analog weights to the CIM module comprises converting the digital weight references in the digital weight storage unit to analog values for storage in the CIM module;
sending, from the digital weight storage unit, the analog values to the CIM module for storage;
applying the plurality of analog weights to process input data via the CIM module using at least one multiply-accumulate operation to generate an output; and
reprogramming one or more analog weights of the plurality of analog weights that are degraded based on corresponding digital weight references stored in the digital weight storage unit,
wherein:
the plurality of digital weight references in the digital weight storage unit are populated with values from a host processing device,
degraded analog weights in the CIM module are reprogrammed based on the corresponding digital weight references from the digital weight storage unit without further reference to the host processing device, wherein the degraded analog weights comprise a weight that causes a digital result of a mathematical operation based on the weight and a defined input to diverge from an expected digital result by a threshold amount, and
the reprogramming is based on detection of an error based on a check bit changing state for a corresponding digital weight reference and correction of the error based on nulling the corresponding digital weight reference.
17. The method of claim 16 , wherein storing the plurality of digital weight references comprises storing the digital weight references corresponding to a selected subset of the plurality of analog weights.
18. A system for performing compute in memory (CIM) computations, comprising:
means for applying a plurality of analog weights to input data using at least one multiply-accumulate operation to generate an output;
means for storing a plurality of digital weight references, wherein a digital weight reference of the plurality of digital weight references corresponds to an analog weight of the plurality of analog weights applied by the means for applying;
means for programming the plurality of analog weights to the means for applying based on the plurality of digital weight references, wherein the means for programming is configured to program the plurality of analog weights to the means for storing a plurality of digital weight references by converting the digital weight references to analog values for storage in the means for storing a plurality of digital weight references, wherein the means for storing a plurality of digital weight references is configured to send the analog values to the means for applying a plurality of analog weights for storage; and
means for reprogramming one or more analog weights of the plurality of analog weights that are degraded based on corresponding digital weight references stored in the means for storing without further reference to a host processing device serving as a source of the digital weight references, wherein the degraded analog weights comprise a weight that causes a digital result of a mathematical operation based on the weight and a defined input to diverge from an expected digital result by a threshold amount, and wherein the means for reprogramming comprises means for detecting an error based on a check bit changing state for a corresponding digital weight reference and means for correcting the error based on nulling the corresponding digital weight reference.