Compensation for conductance drift in analog memory in crossbar array
View Patent ↗A system can compensate for activation drift in analog memory-based artificial neural networks. A set of input activation vectors can be input, at a first point in time, to a crossbar array. The first set of output activation vectors can be read from the output lines of the crossbar array. At a second point in time, which is a later time than the first point in time, the input set of activation vectors can be input to the crossbar array. A second set of output activation vectors can be read from the crossbar array. A function that maps the second set of output activation vectors to the first set of output activation vectors can be determined. The function can be applied to subsequent output activation vectors output by the crossbar array. A method thereof, can also be provided.
1. A method comprising:
inputting, at a first point in time, a set of input activation vectors to a crossbar array, the crossbar array including at least a plurality of input lines, a plurality of output lines, and at least one memory device at each cross point of the plurality of input lines and the plurality of output lines, the at least one memory device at each cross point storing a synaptic weight;
reading out a first set of output activation vectors at the plurality of output lines of the crossbar array, the first set of output activation vectors representing outputs of operations performed on the crossbar array based on the set of input activation vectors input at the first point in time and the synaptic weight stored on the at least one memory device at each cross point;
inputting, at a second point in time, the input set of activation vectors to the crossbar array, the second point in time being a later time than the first point in time;
reading out a second set of output activation vectors at the plurality of output lines of the crossbar array, the second set of output activation vectors representing outputs of the operations performed on the crossbar array based on the set of input activation vectors input at the second point in time and the synaptic weight stored on the at least one memory device at each cross point, wherein the set of input activation vectors applied to the input lines of the crossbar array and used by the crossbar array in computing the first set of output activation vectors, are used by the crossbar array in computing the second set of output activation vectors;
determining a function that maps the second set of output activation vectors which has been read out, to the first set of output activation vectors which has been read out; and
compensating conductance drift exhibited in the crossbar array by applying the function to subsequent output activation vectors output by the crossbar array.
2. The method of claim 1 , wherein the operations include multiply-accumulate operations.
3. The method of claim 1 , wherein the set of input activation vectors, which is input into the crossbar array, is encoded as electrical pulse durations.
4. The method of claim 1 , wherein the set of input activation vectors, which is input into the crossbar array, is encoded as voltage signals.
5. The method of claim 1 , wherein the at least one memory device at each cross point includes an analog non-volatile memory device.
6. The method of claim 1 , wherein the set of input activation vectors are sampled from a machine learning training dataset.
7. The method of claim 1 , wherein the function includes a first-order polynomial least square fit between the first set of output activation vectors and the second set of output activation vectors.
8. The method of claim 1 , wherein the determining a function that maps the second set of output activation vectors to the first set of output activation vectors, includes determining a first-order polynomial least square fit for each column of the crossbar array.
9. The method of claim 1 , wherein the function includes an n-th order polynomial least square fit between the first set of output activation vectors and the second set of output activation vectors, wherein n is an integer greater than 1.
10. The method of claim 1 , wherein the determining a function that maps the second set of output activation vectors to the first set of output activation vectors, includes determining an n-th order polynomial least square fit for each column of the crossbar array, wherein n is an integer greater than 1.
11. The method of claim 1 , wherein the function includes an n-th order polynomial L1 norm fit between the first set of output activation vectors and the second set of output activation vectors, wherein n is an integer greater than 1.
12. The method of claim 1 , wherein the determining a function that maps the second set of output activation vectors to the first set of output activation vectors, includes determining an n-th order polynomial L1 norm fit for each column of the crossbar array, wherein n is an integer greater than 1.
13. The method of claim 1 , wherein the function includes an n-th order polynomial norm fit between the first set of output activation vectors and the second set of output activation vectors, wherein n is an integer greater than 1.
14. The method of claim 1 , wherein the determining a function that maps the second set of output activation vectors to the first set of output activation vectors, includes determining an n-th order polynomial norm fit for each column of the crossbar array, wherein n is an integer greater than 1.
15. The method of claim 1 , wherein the function includes an n-th order polynomial norm fit between the first set of output activation vectors and the second set of output activation vectors, wherein n is an integer greater than or equal to zero.
16. A system comprising:
at least one processor; and
at least one crossbar array arranged with at least a plurality of input lines, a plurality of output lines, and at least one memory device at each cross point of the plurality of input lines and the plurality of output lines, the at least one memory device at each cross point storing a synaptic weight,
the at least one processor configured to:
input, at a first point in time, a set of input activation vectors to the crossbar array;
read first set of output activation vectors from the plurality of output lines of the crossbar array, the first set of output activation vectors representing outputs of operations performed on the crossbar array based on the set of input activation vectors input at the first point in time and the synaptic weight stored on the at least one memory device at each cross point;
input, at a second point in time, the input set of activation vectors to the crossbar array, the second point in time being a later time than the first point in time;
read a second set of output activation vectors from the plurality of output lines of the crossbar array, the second set of output activation vectors representing outputs of the operations performed on the crossbar array based on the set of input activation vectors input at the second point in time and the synaptic weight stored on the at least one memory device at each cross point, wherein the set of input activation vectors applied to the input lines of the crossbar array and used by the crossbar array in computing the first set of output activation vectors, are used by the crossbar array in computing the second set of output activation vectors;
determine a function that maps the second set of output activation vectors which has been read out, to the first set of output activation vectors which has been read out; and
compensating conductance drift exhibited in the crossbar array by apply the function to subsequent output activation vectors output by the crossbar array.
17. The system of claim 16 , wherein the operations include multiply-accumulate operations.
18. The system of claim 16 , wherein the set of input activation vectors, which is input into the crossbar array, is encoded as electrical pulse durations.
19. The system of claim 16 , wherein the set of input activation vectors, which is input into the crossbar array, is encoded as voltage signals.
20. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions readable by a device to cause the device to:
input, at a first point in time, a set of input activation vectors to a crossbar array, the crossbar array including at least a plurality of input lines, a plurality of output lines, and at least one memory device at each cross point of the plurality of input lines and the plurality of output lines, the at least one memory device at each cross point storing a synaptic weight;
read first set of output activation vectors from the plurality of output lines of the crossbar array, the first set of output activation vectors representing outputs of operations performed on the crossbar array based on the set of input activation vectors input at the first point in time and the synaptic weight stored on the at least one memory device at each cross point;
input, at a second point in time, the input set of activation vectors to the crossbar array, the second point in time being a later time than the first point in time;
read a second set of output activation vectors from the plurality of output lines of the crossbar array, the second set of output activation vectors representing outputs of the operations performed on the crossbar array based on the set of input activation vectors input at the second point in time and the synaptic weight stored on the at least one memory device at each cross point, wherein the set of input activation vectors applied to the input lines of the crossbar array and used by the crossbar array in computing the first set of output activation vectors, are used by the crossbar array in computing the second set of output activation vectors;
determine a function that maps the second set of output activation vectors which has been read out, to the first set of output activation vectors which has been read out; and
compensate conductance drift exhibited in the crossbar array by applying the function to subsequent output activation vectors output by the crossbar array.