Counter based resistive processing unit for programmable and reconfigurable artificial-neural-networks
Technical solutions are described for storing weight in a crosspoint device of a resistive processing unit (RPU) array. An example system includes a crosspoint array, wherein each array node represents a connection between neurons of the neural network, and wherein each node stores a weight assigned to the node. The crosspoint array includes a crosspoint device at each node. The crosspoint device includes a counter that has multiple single bit counters, and states of the counters represent the weight to be stored at the crosspoint device. Further, the crosspoint device includes a resistor device that has multiple resistive circuits, and each resistive circuit is associated with a respective counter from the counters. The resistive circuits are activated or deactivated according to a state of the associated counter, and an electrical conductance of the resistor device is adjusted based at least in part on the resistive circuits that are activated.
1. A system for implementing a neural network, the system comprising:
a crosspoint array comprising a plurality of nodes, each node representing a connection between neurons of the neural network, and each node storing a weight assigned to the node, the crosspoint array comprising:
a crosspoint device at each node, the crosspoint device comprising:
a counter comprising a plurality of single bit counters, states of the single bit counters representing the weight to be stored at the crosspoint device; and
a resistor device comprising a plurality of resistive circuits, each resistive circuit associated with a respective single bit counter from the plurality of single bit counters, the resistive circuits are activated or deactivated according to a state of the associated single bit counter, and an electrical conductance of the resistor device being adjusted based at least in part on the resistive circuits that are activated; and
a global controller configured to adjust values stored at each crosspoint device in the crosspoint array.
2. The system of claim 1 , wherein the crosspoint array further comprises:
a local memory at each crosspoint, the local memory configured to store a plurality of weights assigned to the crosspoint device at the crosspoint, each of the plurality of weights corresponding to respective layer-to-layer connection from the neural network;
a local controller at each crosspoint, the local controller coupled with the local memory, the local controller configured to:
read a weight from the local memory; and
set the weight as a value of the counter of the crosspoint device at the crosspoint.
3. The system of claim 2 , wherein the local controller is further configured to store the value of the counter of the crosspoint device to the local memory.
4. The system of claim 1 , wherein the plurality of resistive circuits in the resistor device of the crosspoint device include a quadratically increasing resistance, a first resistive circuit having a predetermined resistance and each further successive resistive circuit of the plurality of resistive circuits having a resistance that is twice of a previous resistive circuit.
5. The system of claim 4 , wherein the plurality of resistive circuits are field effect transistors (FETs).
6. The system of claim 4 , wherein the plurality of resistive circuits are a resistor ladder, each of the resistive circuit comprising:
a logic gate; and
a series of resistors; and
wherein, the logic gate is activated based at least in part on the state of the corresponding single bit counter.
7. A crosspoint array for implementing a neural network, the crosspoint array comprising:
a plurality crosspoint devices, a crosspoint device at respective node of the crosspoint array, each node representing a connection between neurons of the neural network, and each node storing a weight assigned to the node, the crosspoint device comprising:
a counter comprising a plurality of single bit counters, states of the single bit counters representing the weight to be stored at the crosspoint device; and
a resistor device comprising a plurality of resistive circuits, each resistive circuit associated with a respective single bit counter from the plurality of single bit counters, the resistive circuits are activated or deactivated according to a state of the associated single bit counter, and an electrical conductance of the resistor device being adjusted based at least in part on the resistive circuits that are activated.
8. The crosspoint array of claim 7 , further comprising:
a local memory at each crosspoint, the local memory configured to store a plurality of weights assigned to the crosspoint device at the crosspoint, each of the plurality of weights corresponding to respective layer-to-layer connections from the neural network;
a local controller at each crosspoint, the local controller coupled with the local memory, the local controller configured to:
read a weight from the local memory; and
set the weight as a value of the counter of the crosspoint device at the crosspoint.
9. The crosspoint array of claim 8 , wherein the local controller is further configured to store the value of the counter of the crosspoint device to the local memory.
10. The crosspoint array of claim 8 , wherein the plurality of resistive circuits in the resistor device of the crosspoint device include a quadratically increasing resistance, a first resistive circuit of the plurality of resistive circuits having a predetermined resistance and each further successive resistive circuit of the plurality of resistive circuits having a resistance that is twice of a previous resistive circuit.
11. The crosspoint array of claim 10 , wherein the plurality of resistive circuits are field effect transistors (FETs).
12. The crosspoint array of claim 10 , wherein the plurality of resistive circuits are a resistor ladder, each of the resistive circuit comprising:
a logic gate; and
a series of resistors; and
wherein, the logic gate is activated based at least in part on the state of the corresponding single bit counter.