Systems and methods for partial digital retraining
Provided herein is an integrated circuit including, in some embodiments, a hybrid neural network including a plurality of analog layers, a digital layer, and a plurality of data outputs. The plurality of analog layers is configured to include programmed weights of the neural network for decision making by the neural network. The digital layer, disposed between the plurality of analog layers and the plurality of data outputs, is configured for programming to compensate for weight drifts in the programmed weights of the neural network, thereby maintaining integrity of the decision making by the neural network. Also provided herein is a method including, in some embodiments, programming the weights of the plurality of analog layers; determining the integrity of the decision making by the neural network; and programming the digital layer of the neural network to compensate for the weight drifts in the programmed weights of the neural network.
1. A neuromorphic integrated circuit, comprising: a multi-layered analog-digital hybrid neural network comprising a plurality of analog layers configured to include synaptic weights between neural nodes of the neural network for decision making by the neural network; wherein a positively weighted product may be stored in a first column of an analog multiplier array, and a negatively weighted product can be stored in a second column of the analog multiplier array; and at least one digital layer; wherein the digital layer is configured for programmatically compensating for weight drifts of the synaptic weights of the neural network, thereby maintaining integrity of the decision making by the neural network; and wherein the plurality of analog layers is disposed between a plurality of data inputs and the digital layer, and wherein the digital layer is disposed between the plurality of analog layers and a plurality of data outputs, wherein the positively and negatively weighted products can be taken as a differential current value.
2. The neuromorphic integrated circuit of claim 1 , wherein the neural network is configured for one or more classification problems, one or more regression problems, or a combination thereof.
3. The neuromorphic integrated circuit of claim 2 , wherein the decision making by the neural network includes predicting continuous quantities for one or more regression problems.
4. The neuromorphic integrated circuit of claim 2 , wherein the decision making by the neural network includes predicting discrete classes for one or more classification problems.
5. The neuromorphic integrated circuit of claim 1 , further comprising: a test data generator configured to generate test data on a periodic basis for testing the integrity of the decision making by the neural network.
6. The neuromorphic integrated circuit of claim 1 , wherein the analog multiplier array of two-quadrant multipliers in a memory sector of the neuromorphic integrated circuit.
7. The neuromorphic integrated circuit of claim 1 , wherein the neuromorphic integrated circuit is configured to operate on battery power.
8. A neuromorphic integrated circuit, comprising: a multi-layered analog-digital hybrid neural network including a plurality of analog layers configured to include synaptic weights between neural nodes of the neural network for decision making by the neural network, and at least one digital layer, wherein a positively weighted product may be stored in a first column of an analog multiplier array, and a negatively weighted product can be stored in a second column of the analog multiplier array; and wherein the digital layer is configured for programmatically compensating for weight drifts of the synaptic weights of the neural network, thereby maintaining integrity of the decision making by the neural network; a test data generator configured to generate test data on a periodic basis for testing the integrity of the decision making by the neural network; and wherein the plurality of analog layers is disposed between a plurality of data inputs and the digital layer, and wherein the digital layer is disposed between the plurality of analog layers and a plurality of data outputs.
9. The neuromorphic integrated circuit of claim 8 , wherein the decision making by the neural network includes predicting continuous quantities for one or more regression problems.
10. The neuromorphic integrated circuit of claim 8 , wherein the decision making by the neural network includes predicting discrete classes for one or more classification problems.
11. The neuromorphic integrated circuit of claim 8 , wherein the analog multiplier array comprises two-quadrant multipliers in a memory sector of the neuromorphic integrated circuit.
12. The neuromorphic integrated circuit of claim 8 , wherein the neuromorphic integrated circuit is configured for one or more application specific standard products (“ASSPs”) selected from keyword spotting, voice recognition, one or more audio filters, speech enhancement, gesture recognition, image recognition, video object classification and segmentation, and autonomous vehicles including drones.
13. The neuromorphic integrated circuit of claim 12 , wherein the neuromorphic integrated circuit is configured to operate on battery power.
14. A method for a neuromorphic integrated circuit, comprising: programming synaptic weights of a plurality of analog layers of a multi-layered analog-digital hybrid neural network of the neuromorphic integrated circuit for decision making by the neural network; periodically testing an integrity of the decision making by the neural network using test data generated by a test data generator of the neuromorphic integrated circuit; programming a digital layer disposed after a last analog layer of the neural network to compensate for weight drifts in the synaptic weights of the analog layers of the neural network, thereby maintaining integrity of the decision making by the neural network; and wherein a positively weighted product may be stored in a first column of an analog multiplier array, and a negatively weighted product can be stored in a second column of the analog multiplier array.
15. The method of claim 14 , further comprising: predicting continuous quantities for one or more regression problems with the neural network.
16. The method of claim 14 , further comprising: predicting discrete classes for one or more classification problems with the neural network.