Conductance mapping technique for neural networks
Various implementations described herein are directed to a device having neural network circuitry with an array of synapse cells arranged in columns and rows. The device may have input circuitry that provides voltage to the synapse cells by way of row input lines for the rows in the array. The device may have output circuitry that receives current from the synapse cells by way of column output lines for the columns in the array. Also, conductance for the synapse cells in the array may be determined based on the voltage provided by the input circuitry and the current received by the output circuitry.
1 . A device comprising:
neural network circuitry having an array of synapse cells that are arranged in columns and rows;
input circuitry that provides voltage to the synapse cells by way of row input lines for the rows in the array; and
output circuitry that receives current from the synapse cells by way of column output lines for the columns in the array,
wherein conductance for the synapse cells in the array is determined based on the voltage provided by the input circuitry and the current received by the output circuitry,
wherein the conductance for the synapse cells in the array is mapped based on positional orientation of each synapse cell in the array, and
wherein the array is subdivided into sub-blocks, the sub-blocks being selectively interconnected by passgates coupled to the output circuitry by flying bitlines.
2 . The device of claim 1 , wherein:
the conductance for the synapse cells in the array is mapped at programming time based on positional orientation of each synapse cell in the array.
3 . The device of claim 1 , wherein:
each synapse cell has a programmable resistance value, and
the conductance for each synapse cell is calculated and mapped based on the programmable resistance value for each synapse cell.
4 . The device of claim 1 , wherein:
the conductance of the synapse cells includes parasitic conductance based on the positional orientation of the synapse cells such that the conductance is selectively tuned by adjusting a programmable weight of each synapse cell, and
the parasitic conductance is cumulative along row lengths of the row input lines for the rows and along column lengths of the column output lines for the columns between the input circuitry and the output circuitry.
5 . The device of claim 4 , wherein:
the parasitic conductance for the synapse cells is selectively modified by adjusting a resistance value associated with the synapse cells, and
accumulation of the parasitic conductance is selectively modified by adjusting the resistance values associated with the synapse cells along the row lengths of the row input lines for the rows and the column lengths of the column output lines for the columns between the input circuitry and the output circuitry.
6 . The device of claim 1 , wherein:
the conductance of the synapse cells includes parasitic conductance based on one or more characteristics of the synapse cells including positional orientation, conductance drift, temperature and input amplitudes of the synapse cells.
7 . The device of claim 1 , wherein:
the synapse cells are positioned in the array at crossbar intersection points of the columns and the rows.
8 . The device of claim 7 , wherein:
the flying bitlines are used to couple the sub-blocks to the output circuitry so as to receive current from the synapse cells by way of the column output lines for the columns in the array.
9 . The device of claim 1 , wherein:
the synapse cells are programmable synapse cells for use in non-volatile memory (NVM) applications including at least one of resistive random access memory (RRAM) applications, magnetic random access memory (MRAM), and correlated-electron random access memory (CeRAM).
10 . The device of claim 1 , wherein:
the positional orientation of the synapse cells is defined by points of intersection for the rows and columns in the array, and
the conductance for the synapse cells is mapped corresponding to the points of intersection for the rows and columns in the array.
11 . The device of claim 10 , wherein:
the points of intersection refer to resistive-switching (RS) crossbars, and
the positional orientation of the synapse cells refers to the location of the synapse cells in the array at the RS crossbars.
12 . The device of claim 1 , wherein:
the input circuitry comprises digital-to-analog conversion (DAC) circuitry, and
the DAC circuitry is configured to receive digital voltage signals as input, convert the digital voltage signals to analog voltage signals, and then provide the analog voltage signals to the synapse cells by way of the row input lines.
13 . The device of claim 1 , wherein:
the output circuitry comprises analog-to-digital conversion (ADC) circuitry, and
the ADC circuitry is configured to receive analog current signals from the synapse cells by way of the column output lines, convert the analog current signals to digital current signals, and then provide the digital current signals as output.
14 . A method comprising:
providing a neural network with multiple layers such that each layer has synapse cells arranged in an array;
for each layer in the neural network, unrolling weights of the layers following a pre-determined approach, wherein the unrolling comprises converting weight values to conductance values using digital-to-analog converter (DAC) circuitry and measuring the conductance values using analog-to-digital converter (ADC) circuitry;
iterating through the layers to find at least one of an upper boundary for throughput and an upper boundary for utilization for the synapse cells of each layer of the neural network, the upper boundary for throughput and the upper boundary for utilization for the synapse cells of each layer of the neural network being determined by selectively activating sub-blocks of the array via passgates and flying bitlines; and
estimating target conductances and finding a lower boundary for deviation from a target conductance and a real conductance for the synapse cells of each layer of the neural network, wherein the real conductance for the synapse cells of each layer of the neural network includes contributions from position-dependent parasitic effects.
15 . The method of claim 14 , further comprising:
for each layer in the neural network, and for each synapse cell in each layer, compute a final conductance based one or more of synapse cell location, technology parasitics, and state; and
program the target conductances so as to reduce error.
16 . The method of claim 14 , wherein the predetermined approach refers to a weight-to-conductance technique.
17 . A method comprising:
providing a neural network with multiple layers such that each layer has synapse cells arranged in an array;
performing a backward stage on the neural network;
performing a forward stage on the neural network; and
if a neural architecture search (NAS) is used to alter layer characteristics of the multiple layers, then weight training is used to provide a reduction of an effect of parasitics on the neural network by including information from a compute-in-memory structure into the neural network.
18 . The method of claim 17 , further comprising:
if the parasitics degrade accuracy over a threshold, then each layer is split so that a dynamic range of conductances for the synapse cells in each layer is maintained without degradation even when throughput of the neural network is reduced,
wherein information related to parasitics for each synapse cell in each layer is used during layer-to-crossbar mapping so as to reduce the effect of the parasitics on the neural network.
19 . The method of claim 17 , further comprising:
estimating target conductances and reducing the effect of errors by finding a lower boundary for deviation from a target conductance and a real conductance; and
providing the estimated target conductances to the backward stage so that the backward stage is performed based on the target conductances.