Compute in-memory architecture for continuous on-chip learning
A system capable of providing on-chip learning comprising a processor and a plurality of compute engines coupled with the processor. Each of the compute engines including a compute-in-memory (CIM) hardware module and a local update module. The CIM hardware module stores a plurality of weights corresponding to a matrix and is configured to perform a vector-matrix multiplication for the matrix. The local update module is coupled with the CIM hardware module and configured to update at least a portion of the weights.
1 . A system, comprising:
a processor; and
a plurality of compute engines coupled with the processor, the plurality of compute engines including a compute-in-memory (CIM) hardware module, a bit mixer, and a local update module, the CIM hardware module storing a plurality of weights corresponding to a matrix and configured to perform a vector-matrix multiplication (VMM) for the matrix, the local update module being coupled with the CIM hardware module and configured to update at least a portion of the plurality of weights,
wherein:
the bit mixer is coupled to the CIM and configured to combine results from the CIM into a weighted output by using exponential weights and bit slicing;
the CIM hardware module includes a plurality of cells for storing the plurality of weights; and
the local update module includes:
an adder configured to be selectively coupled with each of the plurality of cells, to receive a weight update, and to add the weight update with a weight of the plurality of weights for each of the plurality of cells; and
write circuitry coupled with the adder and the plurality of cells, the write circuitry configured to write a sum of the weight and the weight update to each of the plurality of cells, and
a compute engine of the plurality of compute engines includes:
a controller configured to provide a plurality of control signals to the CIM hardware module and the local update module,
a first portion of the plurality of control signals corresponding to an inference mode, a second portion of the plurality of control signals corresponding to a weight update mode,
wherein the weight update mode includes providing the weight update to the adder.
2 . The system of claim 1 , wherein the plurality of cells is selected from a plurality of analog static random access memory (SRAM) cells, a plurality of digital SRAM cells, and a plurality of resistive random access memory (RRAM) cells.
3 . The system of claim 2 , wherein the plurality of cells includes the plurality of analog SRAM cells, the CIM hardware module further including a capacitive voltage divider for each of the plurality of analog SRAM cells.
4 . The system of claim 1 , wherein the local update module further includes: a local batched weight update calculator coupled with the adder and configured to determine the weight update.
5 . The system of claim 1 , wherein each of the plurality of compute engines further includes: address circuitry configured to selectively couple the adder and the write circuitry with any of the plurality of cells.
6 . The system of claim 1 , wherein the plurality of weights includes at least one positive weight and at least one negative weight.
7 . The system of claim 1 , further comprising:
a scaled vector accumulation (SVA) unit coupled with the plurality of compute engines and the processor, the SVA unit configured to apply an activation function to an output of the plurality of compute engines.
8 . The system of claim 7 , wherein the SV A unit and the plurality of compute engines are in a plurality of tiles.
9 . The system of claim 1 , wherein the bit mixer is coupled to an analog-to-digital converter (ADC) that is coupled to an output cache, the output cache being configured to pass data by rows.
10 . The system of claim 1 , wherein the CIM module is configured to perform the VMM by mapping the plurality of weights to bipolar weights and multiplying the matrix with an input vector and scaling the multiplication result based on a symmetric range.
11 . The system of claim 1 , wherein the compute engine is further configured to comprise an input cache, a control unit, and an output cache.
12 . A machine learning system, comprising:
at least one processor; and
a plurality of tiles coupled with the at least one processor, the plurality of tiles including a plurality of compute engines and at least one scaled vector accumulation (SVA) unit, the SVA unit configured to apply an activation function to an output of the plurality of compute engines the plurality of compute engines being interconnected and coupled with the SV A unit, each of the plurality of compute engines including at least one compute-in-memory (CIM) hardware module, a controller, a bit mixer and at least one local update module, the at least one CIM hardware module including a plurality of static random access memory (SRAM) cells storing a plurality of weights corresponding to a matrix, the at least one CIM hardware module being configured to perform a vector-matrix multiplication (VMM) for the matrix, the at least one local update module being coupled with the at least one CIM hardware module and configured to update at least a portion of the plurality of weights, the controller being configured to provide a plurality of control signals to the at least one CIM hardware module and the at least one local update module, a first portion of the plurality of control signals corresponding to an inference mode, a second portion of the plurality of control signals corresponding to a weight update mode, wherein:
the bit mixer is coupled to the CIM and configured to combine results from the CIM into a weighted output by using exponential weights and bit slicing; and
a local update module of the at least one local update module includes:
an adder configured to be selectively coupled with each of the plurality of SRAM cells, to receive a weight update, and to add the weight update with a weight of the plurality of weights for each of the plurality of SRAM cells; and
write circuitry coupled with the adder and the plurality of SRAM cells, the write circuitry configured to write a sum of the weight and the weight update to each of the plurality of SRAM cells, wherein the weight update mode includes providing the weight update to the adder.
13 . The machine learning system of claim 12 , wherein each of the plurality of compute engines further includes address circuitry configured to selectively couple the adder and the write circuitry with each of the plurality of SRAM cells.
14 . A method, comprising:
providing an input vector to a plurality of compute engines coupled with a processor, the plurality of compute engines including a compute-in-memory (CIM) hardware module, a bit mixer, and a local update module, the CIM hardware module storing a plurality of weights corresponding to a matrix in a plurality of cells and configured to perform a vector-matrix multiplication (VMM), the local update module being coupled with the CIM hardware module and configured to update at least a portion of the plurality of weights, wherein the bit mixer is configured to combine results from the CIM into a weighted output by using exponential weights and bit slicing;
performing the VMM of the input vector and the matrix using the plurality of compute engines;
determining at least one weight update for the plurality of weights; and
locally updating the plurality of weights using the at least one weight update and the local update module, wherein:
a compute engine of the plurality of compute engines includes:
a controller configured to provide a plurality of control signals to the CIM hardware module and the local update module, a first portion of the plurality of control signals corresponding to an inference mode, a second portion of the plurality of control signals corresponding to a weight update mode, wherein the weight update mode includes providing the at least one weight update to an adder; and the locally updating of the plurality of weights includes:
adding, using the adder configured to be selectively coupled with each of the plurality of cells, the at least one weight update to a weight of at least a portion of the plurality of weights for each of the plurality of cells; and
writing, using write circuitry coupled with the adder and the plurality of cells, a sum of the weight and the weight update to each of the plurality of cells.
15 . The method of claim 14 , wherein the plurality of cells is selected from a plurality of analog static random access memory (SRAM) cells, a plurality of digital SRAM cells, and a plurality of resistive random access memory (RRAM) cells.
16 . The method of claim 14 , wherein the plurality of weights includes at least one positive weight and at least one negative weight.
17 . The method of claim 14 , further comprising: applying an activation function to an output of the plurality of compute engines.
18 . The method of claim 17 , wherein the applying further includes: using a scaled vector accumulation (SV A) unit coupled with the plurality of compute engines to apply the activation function to the output of the plurality of compute engines.
19 . The method of claim 14 , wherein the bit mixer is coupled with the CIM module and an analog-to-digital converter (ADC) that is coupled to an output cache, the output cache being configured to pass data by rows.
20 . The method of claim 14 , wherein the CIM module is configured to perform the VMM by mapping the plurality of weights to bipolar weights and multiplying the matrix with an input vector and scaling the multiplication result based on a symmetric range.