Systems and methods for harnessing analog noise in efficient optimization problem accelerators
Systems and methods are provided for implementing a hardware accelerator. The hardware accelerator emulates a neural network, and includes a memristor crossbar array, and a non-linear filter. The memristor crossbar array can be programmed to calculate node values of the neural network. The nodes values can be calculated in accordance with rules to reduce an energy function associated with the neural network. The non-linear filter is coupled to the memristor crossbar array and programmed to harness noise signals that may be present in analog circuitry of the hardware accelerator. The noise signals can be harnessed such that the energy function associated with the neural network converges towards a global minimum and modifies the calculated node values. In some embodiments, the non-liner filter is implemented as a Schmidt trigger comparator.
1. A circuit, comprising:
an array of analog elements programmed to calculate node values of a neural network, wherein the nodes values are output by the array of analog elements and calculated in accordance with rules to reduce an energy function associated with the neural network; and
a circuit element coupled to the array of analog elements, the circuit element programmed to permit a noise signal to effect the calculated node values such that the energy function associated with the neural network converges towards a global minimum and modifies the calculated node values output by the array of analog elements, wherein the circuit element comprises at least one of: a Schmidt trigger comparator and a low pass filter.
2. The circuit of claim 1 , wherein the Schmidt trigger comparator is programmed to permit the noise signal to effect the calculated node values output by the array of analog elements in accordance with a tolerance to an amount of the noise signal for triggering a modification to the calculated node values.
3. The circuit of claim 2 , wherein the Schmidt trigger comparator is programmed to permit the noise signals to effect the calculated node values output by the array of analog elements such that the noise signals are greater than an optimal noise value associated with the neural network.
4. The circuit of claim 3 , wherein the Schmidt trigger comparator is programmed to dynamically adjust the tolerance to an amount of the noise signal for triggering a modification to the calculated node values output by the array of analog elements as a function of time.
5. The circuit of claim 4 , wherein the Schmidt trigger comparator is programmed to gradually increase the tolerance to an amount of the noise signal for triggering a modification to the calculated node values output by the array of analog elements as a function of increasing time.
6. The circuit of claim 5 , wherein the Schmidt trigger comparator is programmed to dynamically adjust the tolerance to an amount of the noise signal for triggering a modification to the calculated node values output by the array of analog elements by adjusting a width between an upper trigger limit and a lower trigger limit.
7. The circuit of claim 1 , wherein the noise signal comprises a noise signal intrinsically generated by the array of analog elements.
8. The circuit of claim 1 , wherein the low pass filter is programmed to permit the noise signal to effect the calculated node values output by the array of analog elements in accordance with an amount of the noise signal that is filtered.
9. The circuit of claim 8 , wherein the low pass filter is programmed to dynamically adjust the amount of the noise signal that is filtered as a function of time.
10. The circuit of claim 1 , wherein the array of array of analog elements comprises one of: a memristor crossbar array, a memristor bipolar array, phase-change memory (PCM), Flash cells, and magnetoresistive random access memory (MRAM).
11. The circuit of claim 1 , wherein the modified node values are updated in accordance with an update rule to generate new inputs values into the neural network for a calculation of new node values.
12. The circuit of claim 11 , comprising a controller is programmed to:
determine whether the new node values are final node values of the neural network; and
in response to determining that the new node values are not final node values, mapping the new input values to input voltages to be received as input by the array of analog elements.
13. The circuit of claim 12 , where the controller is further programmed to:
in response to determining that the new node values are the final node values, the new node values correspond to a solution of a computational problem mapped to the neural network.
14. The circuit of claim 1 , wherein the neural network comprises a Hopfield Neural Network (HNN).
15. A hardware accelerator, comprising:
a memristor crossbar array programmed to calculate node values of a neural network, wherein the nodes values are output by the memristor crossbar array and calculated in accordance with rules to reduce an energy function associated with the neural network; and
a non-linear filter coupled to the memristor crossbar array programmed to permit a noise signal to effect the calculated node values such that the energy function associated with the neural network converges towards a global minimum and modifies the calculated node values output by the memristor crossbar array, wherein the non-linear filter comprises at least one of: a Schmidt trigger comparator and a low pass filter.
16. The hardware accelerator of claim 15 , wherein the Schmidt trigger comparator is programmed to permit the noise signal to effect the calculated node values output by the memristor crossbar array in accordance with a tolerance to an amount of the noise signal for triggering a modification to the calculated node values.
17. The hardware accelerator of claim 16 , wherein the Schmidt trigger comparator is programmed to dynamically adjust the tolerance to an amount of the noise signal for triggering a modification to the calculated node values output by the memristor crossbar array as a function of time.
18. The hardware accelerator of claim 15 , wherein the low pass filter is programmed to permit the noise signal to effect the calculated node values output by the array of analog elements in accordance with an amount of the noise signal that is filtered.
19. The hardware accelerator of claim 15 , wherein the noise signal comprises a noise signal intrinsically generated by the memristor crossbar array.
20. The hardware accelerator claim 15 , wherein the memristor crossbar array comprises:
a plurality of weight row lines;
a plurality of weight column lines; and
a plurality of memory cells, wherein each of the plurality of memory cells are coupled across an intersection of one of the plurality of weight row lines and one of the plurality of weight column lines and programmed according to a weight matrix to output the calculated node value of the neural network across each of the plurality of weight column lines.