Mitigation of conductance drift in neural network resistive processing units
Methods and systems for performing calculations with a neural network include determining a conductance drift coefficient for resistive processing unit (RPU) weights in a neural network. A correction factor is applied to neuron inputs in the neural network in accordance with the drift coefficient and a time that has elapsed since the RPU weights were programmed. A calculation is performed with the neural network. The correction factor compensates for conductance drift.
1. A method for performing calculations with a neural network, comprising:
determining a conductance drift coefficient for resistive processing unit (RPU) weights in a neural network;
applying a correction factor to neuron inputs in the neural network in accordance with the drift coefficient and a time that has elapsed since the RPU weights were programmed; and
performing a calculation with the neural network, wherein the correction factor compensates for conductance drift.
2. The method of claim 1 , wherein the conductance drift coefficient is determined according to a distribution of drift coefficients for a plurality of RPU weights.
3. The method of claim 2 , wherein the conductance drift coefficient is determined from the distribution using a statistical function selected from the group consisting of the mode of the distribution and the mean of the distribution.
4. The method of claim 1 , further comprising quantizing values of the RPU weights.
5. The method of claim 4 , further comprising periodically requantizing the values of the RPU weights in accordance with conductance drift.
6. The method of claim 5 , wherein requantization is performed before the conductance drift reaches half of a quantization value.
7. The method of claim 1 , further comprising determining an amount of conductance drift based on the conductance drift coefficient.
8. The method of claim 7 , wherein the correction factor comprises a multiplicative factor that is the inverse of the amount of conductance drift.
9. The method of claim 1 , wherein the RPU weights are phase change memory devices that have a settable conductance value that changes over time.
10. The method of claim 1 , wherein the correction factor is expressed as:
(
1
(
t
t
0
)
-
v
)
where t is a present time, measured from a time of programming, t 0 is a test time, measured at some previous time since programming, and ν is the drift coefficient.
11. A system for performing calculations with a neural network, comprising:
a neural network that includes resistive processing unit (RPU) weights;
a drift correction module configured to determine a conductance drift coefficient for the RPU weights, to apply a correction factor to neuron inputs in the neural network in accordance with the drift coefficient and a time that has elapsed since the RPU weights were programmed, wherein the the correction factor compensates for conductance drift during calculations performed by the neural network.
12. The system of claim 11 , wherein the drift correction module is further configured to determine the conductance drift coefficient according to a distribution of drift coefficients for a plurality of RPU weights.
13. The system of claim 12 , wherein the drift correction module is further configured to determine conductance drift coefficient from the distribution using a statistical function selected from the group consisting of the mode of the distribution and the mean of the distribution.
14. The system of claim 11 , further comprising a weight quantizer configured to quantize values of the RPU weights.
15. The system of claim 14 , wherein the weight quantizer is further configured to requantize the values of the RPU weights in accordance with conductance drift.
16. The system of claim 15 , wherein the weight quantizer is further configured to perform requantization before the conductance drift reaches half of a quantization value.
17. The system of claim 11 , wherein the drift correction module is further configured to determine an amount of conductance drift based on the conductance drift coefficient.
18. The system of claim 17 , wherein the correction factor comprises a multiplicative factor that is the inverse of the amount of conductance drift.
19. The system of claim 11 , wherein the RPU weights are phase change memory devices that have a settable conductance value that changes over time.
20. A non-transitory computer readable storage medium comprising a computer readable program for performing calculations with a neural network, wherein the computer readable program when executed on a computer causes the computer to perform the steps of:
determining a conductance drift coefficient for resistive processing unit (RPU) weights in a neural network;
applying a correction factor to neuron inputs in the neural network in accordance with the drift coefficient and a time that has elapsed since the RPU weights were programmed; and
performing a calculation with the neural network, wherein the correction factor compensates for conductance drift.