Systems and methods for optimizing energy efficiency of analog neuromorphic circuits
Systems and methods are provided for optimizing energy efficiency of analog neuromorphic circuits. The method includes obtaining an integrated circuit implementing an analog network of analog components including operational amplifiers and resistors. The analog network represents a trained neural network, each operational amplifier represents an analog neuron, and each resistor represents a connection between two analog neurons. The method also includes generating inferences using the integrated circuit for test inputs, including simultaneously transferring signals from one layer to a subsequent layer. The method also includes, while generating inferences: in accordance with a determination that a level of signal output of the operational amplifiers is equilibrated: determining an active set of analog neurons of the analog network influencing signal formation for propagation of signals; and turning off power for other analog neurons of the analog network, for a predetermined period of time.
1. A method for optimizing energy efficiency of analog neuromorphic circuits, the method comprising:
obtaining an integrated circuit implementing an analog network of analog components including a plurality of operational amplifiers and a plurality of resistors, wherein:
the analog network represents a trained neural network;
each operational amplifier represents a respective analog neuron;
each resistor represents a respective connection between a respective first analog neuron and a respective second analog neuron; and
each resistor has a fixed resistance value; and
generating inferences using the integrated circuit for a plurality of test inputs, including simultaneously transferring signals from one layer to a subsequent layer of the analog network; and
while generating inferences using the integrated circuit:
determining that a level of signal output of the plurality of operational amplifiers is equilibrated; and
in accordance with the determination that the level of signal output is equilibrated:
determining an active set of analog neurons of the analog network influencing signal formation for propagation of signals; and
turning off power for one or more operational amplifiers corresponding to analog neurons of the analog network, distinct from the active set of analog neurons, for a predetermined period of time.
2. The method of claim 1 , wherein determining the active set of analog neurons is based on calculating delays of signal propagation through the analog network.
3. The method of claim 1 , wherein determining the active set of analog neurons is based on detecting the propagation of signals through the analog network.
4. The method of claim 1 , wherein the trained neural network is a feed-forward neural network, and the active set of analog neurons belong to an active layer of the analog network, and turning off power includes turning off power for one or more layers prior to the active layer of the analog network.
5. The method of claim 1 , wherein the predetermined period of time is calculated based on simulating propagation of signals through the analog network, accounting for signal delays.
6. The method of claim 1 , wherein the trained neural network is a recurrent neural network (RNN), and the analog network further includes one or more analog components other than the plurality of operational amplifiers and the plurality of resistors, the method further comprising:
in accordance with the determination that the level of signal output is equilibrated, turning off power, for the one or more analog components, for the predetermined period of time.
7. The method of claim 1 , further comprising:
turning on power for the one or more operational amplifiers corresponding to analog neurons of the analog network after the predetermined period of time.
8. The method of claim 1 , wherein determining that the level of signal output of the plurality of operational amplifiers is equilibrated is based on detecting if one or more operational amplifiers of the analog network is outputting more than a predetermined threshold signal level.
9. The method of claim 1 , further comprising:
repeating the turning off for the predetermined period of time and turning on the active set of analog neurons for the predetermined period of time, while generating the inferences.
10. The method of claim 1 , further comprising:
in accordance with the determination that the level of signal output is equilibrated, for each inference cycle:
during a first time interval, determining a first layer of analog neurons of the analog network influencing signal formation for propagation of signals;
turning off power for a first one or more analog neurons of the analog network, prior to the first layer, for the predetermined period of time; and
during a second time interval subsequent to the first time interval, turning off power for a second one or more analog neurons including the first layer of analog neurons and the first one or more analog neurons of the analog network, for the predetermined period.
11. The method of claim 1 , wherein the one or more analog neurons consist of analog neurons of a first one or more layers of the analog network, and the active set of analog neurons consist of analog neurons of a second layer of the analog network, and the second layer of the analog network is distinct from layers of the first one or more layers.
12. A system for optimizing energy efficiency of analog neuromorphic circuits, comprising:
one or more processors;
memory;
wherein the memory stores one or more programs configured for execution by the one or more processors, and the one or more programs comprising instructions for:
obtaining an integrated circuit implementing an analog network of analog components including a plurality of operational amplifiers and a plurality of resistors, wherein:
the analog network represents a trained neural network;
each operational amplifier represents a respective analog neuron;
each resistor represents a respective connection between a respective first analog neuron and a respective second analog neuron; and
each resistor has a fixed resistance value; and
generating inferences using the integrated circuit for a plurality of test inputs, including simultaneously transferring signals from one layer to a subsequent layer of the analog network; and
while generating inferences using the integrated circuit:
determining that a level of signal output of the plurality of operational amplifiers is equilibrated; and
in accordance with the determination that the level of signal output is equilibrated:
determining an active set of analog neurons of the analog network influencing signal formation for propagation of signals; and
turning off power for one or more operational amplifiers corresponding to analog neurons of the analog network, distinct from the active set of analog neurons, for a predetermined period of time.
13. The system of claim 12 , wherein determining the active set of analog neurons is based on calculating delays of signal propagation through the analog network.
14. The system of claim 12 , wherein determining the active set of analog neurons is based on detecting the propagation of signals through the analog network.
15. The system of claim 12 , wherein the trained neural network is a feed-forward neural network, and the active set of analog neurons belong to an active layer of the analog network, and turning off power includes turning off power for one or more layers prior to the active layer of the analog network.
16. The system of claim 12 , wherein the predetermined period of time is calculated based on simulating propagation of signals through the analog network, accounting for signal delays.
17. The system of claim 12 , wherein the trained neural network is a recurrent neural network (RNN), and the analog network further includes one or more analog components other than the plurality of operational amplifiers and the plurality of resistors, and wherein the one or more programs further comprise instructions for:
in accordance with the determination that the level of signal output is equilibrated, turning off power, for the one or more analog components, for the predetermined period of time.
18. The system of claim 12 , wherein the one or more programs further comprise instructions for:
turning on power for the one or more analog neurons of the analog network after the predetermined period of time.
19. The system of claim 12 , wherein determining that the level of signal output of the plurality of operational amplifiers is equilibrated is based on detecting if one or more operational amplifiers of the analog network is outputting more than a predetermined threshold signal level.
20. A non-transitory computer readable storage medium storing one or more programs configured for execution by a computer system having one or more processors, the one or more programs comprising instructions for:
obtaining an integrated circuit implementing an analog network of analog components including a plurality of operational amplifiers and a plurality of resistors, wherein:
the analog network represents a trained neural network;
each operational amplifier represents a respective analog neuron;
each resistor represents a respective connection between a respective first analog neuron and a respective second analog neuron; and
each resistor has a fixed resistance value; and
generating inferences using the integrated circuit for a plurality of test inputs, including simultaneously transferring signals from one layer to a subsequent layer of the analog network; and
while generating inferences using the integrated circuit:
determining that a level of signal output of the plurality of operational amplifiers is equilibrated; and
in accordance with the determination that the level of signal output is equilibrated:
determining an active set of analog neurons of the analog network influencing signal formation for propagation of signals; and
turning off power for one or more operational amplifiers corresponding to analog neurons of the analog network, distinct from the active set of analog neurons, for a predetermined period of time.