THREE DIMENSIONAL NAND MULTIPLY AND ACCUMULATE WITH DYNAMIC INPUTS
Multiply and accumulate (MAC) operations typically involve extremely large amounts of data and large numbers of operations. As such, they are extremely computationally intensive, involving large numbers of data transfers and consuming large amounts of time and power. To address these problems, the following presents methods of realizing a MAC engine in a 3D NAND flash die. The engine takes as input two vectors and outputs their dot product. The dot product of two vectors is the building block of matrix multiplication. The 3D NAND MAC engine presented here can be used to implement modern machine learning algorithms, in particular Neural Networks. The two vector operands are not programed into the NAND memory cells, therefore the endurance of the device is not compromised.
1 . An apparatus, comprising:
a control circuit configured to connect to an array of non-volatile memory cells having a three dimensional NAND architecture in which NAND strings extend vertically above a substrate through a plurality of horizontal word line layers, along which memory cells of the NAND strings are connected, and through a select gate layer, along which a select gate of each of the NAND strings is connected, the select gate layer having multiple individually biasable sections corresponding to sub-sets of the NAND strings of the array, the array also having a plurality of bit lines each connected to one of the NAND strings of each of a plurality of the sub-sets of the NAND strings, the control circuit is configured to:
receive a first input vector;
receive a second input vector;
apply the first input vector as a set of bias levels to a plurality of the word line layers;
concurrently with applying the first input vector as the set of bias levels to the plurality of the word line layers, apply the second input vector as a set of bias levels to a first plurality of the individually biasable sections of the select gate layer; and
collect a product of the first input vector, the second input vector, and a first matrix along a first bit line, the first matrix stored as threshold voltage values of a plurality of memory cells on the plurality of word line layers of a NAND string of the NAND strings connected to one of the first plurality of individually biasable sections of the select gate layer and to the first bit line.
2 . The apparatus of claim 1 , wherein the control circuit is formed on a control die, the apparatus further comprising:
a memory die including the array of non-volatile memory cells, the memory die separate from and bonded to the control die.
3 . The apparatus of claim 1 , wherein the control circuit is further configured to:
receive the first matrix; and
program the first matrix into the array as threshold voltages of memory cells.
4 . The apparatus of claim 1 , wherein values of the first matrix are binary values.
5 . The apparatus of claim 1 , wherein the first matrix is an identity matrix and the product of the first input vector, the second input vector, and the first matrix is a dot product of the first input vector and the second input vector.
6 . The apparatus of claim 1 , wherein the first input vector and the second input vector are binary valued.
7 . The apparatus of claim 1 , wherein the first input vector and the second input vector are analog valued.
8 . The apparatus of claim 1 , wherein the control circuit is further configured to:
receive one or more additional second input vectors;
concurrently with applying the first input vector as the set of bias levels to the plurality of the word line layers and applying the second input vector as the set of bias levels to the first plurality of the individually biasable sections of the select gate layer, apply each of the additional second input vectors as a set of bias levels to a corresponding plurality of the individually biasable sections of the select gate layer; and
collect a product of the first input vector with each of the additional second input vectors and an additional matrix along a corresponding bit line, each of the additional matrices stored as threshold voltage values of a plurality of memory cells on the plurality of word line layers of a NAND string of each of the NAND strings connected to a corresponding one of the additional pluralities of individually biasable sections of the select gate layer.
9 . The apparatus of claim 8 , wherein each of the first matrix and the additional matrices is an identity matrix.
10 . The apparatus of claim 1 , wherein the first input vector is either a row or a column of a second matrix.
11 . The apparatus of claim 1 , wherein the second input vector is either a row or a column of a second matrix.
12 . The apparatus of claim 1 , wherein the first input vector and the second input vector respectively weight values of a first set of weights and a second set of weights of neural network.
13 . The apparatus of claim 1 , wherein the control circuit is further configured to:
concurrently with applying the first input vector as the set of bias levels to the plurality of the word line layers and applying the second input vector as the set of bias levels to the first plurality of the individually biasable sections of the select gate layer, biasing the word lines layers other the word lines to which the first input vector is applied to a read bypass voltage.
14 . The apparatus of claim 1 , wherein the control circuit is further configured to:
concurrently with applying the first input vector as the set of bias levels to the plurality of the word line layers and applying the second input vector as the set of bias levels to the first plurality of the individually biasable sections of the select gate layer, biasing the individually biasable sections of the select gate layer of others of the word lines to which the second input vector is applied to a non-select voltage.
15 . A method, comprising:
receiving a matrix of values;
programming the matrix of values into an array of non-volatile memory cells, the array having a three dimensional NAND architecture in which NAND strings extend vertically above a substrate through a plurality of horizontal word line layers, along which memory cells of the NAND strings are connected, and through a select gate layer, along which a select gate of each of the NAND strings is connected, the select gate layer having multiple individually biasable sections corresponding to sub-sets of the NAND strings of the array, the array also having a plurality of bit lines each connected to one of the NAND strings of each of a plurality of the sub-sets of the NAND strings, where a first matrix is programmed into the array as threshold voltage values of a plurality of memory cells on the plurality of word line layers of a NAND string of the NAND strings connected to one of the first plurality of individually biasable sections of the select gate layer and to a first bit line;
receiving a first input vector and a second input vector; and
performing an in-memory computation of a product of the first input vector, the matrix of values, and the second input vector by:
converting the first input vector and the second input vector respectively into a first set of bias levels and a second set of bias levels;
concurrently applying the first set of bias levels to a plurality of the word line layers and applying the second of bias levels to a first plurality of the individually biasable sections of the select gate layer; and
collecting the product of the first input vector, the matrix of values, and the second input vector on the first bit line.
16 . The method of claim 15 , wherein the matrix of values is an identity matrix and the product of the first input vector, the matrix of values, and the second input vector is a dot product of the first input vector and the second input vector.
17 . The method of claim 15 , wherein the first input vector and the second input vector are analog valued.
18 . A non-volatile memory device, comprising:
an array of non-volatile memory cells, comprising:
a plurality of NAND strings extending vertically above a substrate;
a plurality of word line layers extending horizontally above the substrate and along which the memory cells of the NAND strings are connected;
a select gate layer extending horizontally above the substrate having multiple individually biasable sections corresponding to sub-sets of the NAND strings of the array; and
a plurality of bit lines each connected to one of the NAND strings of each of a plurality of the sub-sets of the NAND strings;
one or more control circuits connected to the array of non-volatile memory cells and configured to:
program values of a matrix as threshold voltage value into a plurality of memory cells of a plurality of NAND strings connected along a first bit line, the plurality of memory cells connected to a first plurality of word line layers and each of the plurality of NAND strings connected to one of a first plurality of individually biasable sections;
bias the first plurality of word lines according to a first input vector;
bias the first plurality of individually biasable sections according to a second input vector; and
collect a product of the first input vector, the matrix, and the second input vector on the first bit line.
19 . The non-volatile memory device of claim 18 , wherein the matrix in an identity matrix and the product of the first input vector, the matrix, and the second input vector is a dot product of the first input vector and the second input vector.
20 . The non-volatile memory device of claim 18 , wherein the first input vector and the second input vector respectively weight values of a first set of weights and a second set of weights of neural network.