Non-volatile memory die with deep learning neural network
Exemplary methods and apparatus are provided for implementing a deep learning accelerator (DLA) or other neural network components within the die of a non-volatile memory (NVM) apparatus using, for example, under-the-array circuit components within the die. Some aspects disclosed herein relate to configuring the under-the-array components to implement feedforward DLA operations. Other aspects relate to backpropagation operations. Still other aspects relate to using an NAND-based on-chip copy with update function to facilitate updating synaptic weights of a neural network stored on a die. Other aspects disclosed herein relate to configuring a solid state device (SSD) controller for use with the NVM. In some aspects, the SSD controller includes flash translation layer (FTL) tables configured specifically for use with neural network data stored in the NVM.
1. An apparatus comprising:
non-volatile memory (NVM) NAND elements formed in a memory array including one or more memory dies, the NVM NAND elements including single-bit-per-cell elements and multiple-bit-per-cell elements;
a neural network computing component formed on or in at least one memory die of the memory array and coupled to the NVM NAND elements, the neural network computing component configured to perform neural network computing operations;
a read-modify-write component formed on or in the at least one memory die and configured to obtain neural network training data from a memory external to the memory array, read synaptic weights from the NVM NAND elements of the memory array, update the synaptic weights using the neural network computing component based on the training data, and store the updated synaptic weights in the single-bit-per cell elements of the memory array; and
an on-chip copy with update component formed on or in the at least one memory die and configured to perform an on-chip copy of the updated synaptic weights stored in the NVM NAND elements, wherein the on-chip copy with update component is further configured to perform a fold operation in which the updated synaptic weights are read from a plurality of the single-bit-per-cell elements of the memory array, are further updated in a neural network computing operation using the neural network computing component, and are folded into at least one of the multiple-bit-per-cell elements of the memory array.
2. The apparatus of claim 1 , wherein the neural network computing component is a backpropagation component and wherein the on-chip copy with update component is configured to update the synaptic weights during the on-chip fold operation using the backpropagation component.
3. The apparatus of claim 2 , wherein the backpropagation component comprises: a synaptic weight determination circuit and a synaptic weight update circuit.
4. The apparatus of claim 1 , wherein the on-chip copy and update component is an extra-array processing component formed on or in the at least one memory die.
5. The apparatus of claim 1 , wherein the multiple-bit-per-cell elements comprise at least one of triple-level cell (TLC) elements and quad-level cell (QLC) elements.
6. The apparatus of claim 5 , wherein the on-chip copy with update component is further configured to perform the fold operation by consolidating synaptic weights stored in 3*N SLC elements into modified synaptic weights stored in N TLC elements.
7. The apparatus of claim 5 , wherein the on-chip copy with update component is further configured to perform the fold operation by consolidating synaptic weights stored in 4*N SLC elements into modified synaptic weights stored in N QLC elements.
8. The apparatus of claim 1 , wherein the at least one memory die is configured to store the synaptic weights within a plurality of NAND word lines, and wherein the neural network computing component is further configured to sense the synaptic weights in parallel from the NAND word lines and perform a plurality of the neural network operations in parallel using the synaptic weights.
9. The apparatus of claim 1 , wherein the on-chip copy with update component is further configured to:
receive a physical block identifier from a device that is external to the memory array, that identifies a source block within the memory array that corresponds to the single-bit-per-cell elements;
read the synaptic weights from the single-bit-per-cell elements of the source block of the memory array identified by the physical block identifier received from the external device;
update the synaptic weights of the source block using the neural network computing component that performs the neural network computing operation;
fold updated synaptic weights into multiple-bit-per-cell elements of a target block of the memory array that corresponds to an updated physical block identifier; and
send a command completion response to the external device that notifies the external device that the neural network computing operation has completed and provides the updated physical block identifier of one or more synaptic weights that have been changed by the on-chip copy with update component.
10. The apparatus of claim 1 ,
wherein the NVM NAND elements are configured to store the updated synaptic weights within a plurality of word lines;
wherein the neural network computing component is one of a plurality of neural network computing components formed on or in the least one memory die of the memory array; and
wherein the plurality of neural network computing components are configured to access the updated synaptic weights in parallel from the plurality of word lines and perform the neural network computing operations in parallel using the updated synaptic weight values.
11. A method for neural network processing using a memory array of memory dies that include non-volatile memory (NVM) NAND elements, the method comprising:
sensing synaptic weights of a neural network stored within the NVM NAND elements;
performing a read-modify-write operation comprising obtaining neural network training data from a memory external to the memory array, updating the synaptic weights based on the training data using neural network computing circuits formed on or in at least one memory die of the memory array, and storing the updated synaptic weights in the single-bit-per cell elements of the memory array; and
performing an on-chip copy and update within the memory array, wherein the performing of the on-chip copy and update includes performing a fold operation in which at least some of the updated synaptic weights are sensed from single-bit-per-cell NVM NAND elements of the memory array, are further updated in a neural network computing operation using the neural network computing circuits, and are folded into multiple-bit-per-cell NVM NAND elements of the memory die.
12. The method of claim 11 , wherein performing the fold operation further comprises consolidating synaptic weights stored in a plurality of single-level cell (SLC) elements into modified synaptic weights stored in at least one of triple level cell (TLC) elements and quad-level cell (QLC) elements.
13. The method of claim 12 , wherein performing the fold operation further comprises consolidating synaptic weights stored in 3*N SLC elements into modified synaptic weights stored in N TLC elements.
14. The method of claim 12 , wherein performing the fold operation further comprises consolidating synaptic weights stored in 4*N SLC elements into modified synaptic weights stored in N QLC elements.
15. The method of claim 11 , wherein the memory die is configured to store the synaptic weights within a plurality of NAND word lines, and wherein performing the neural network operation on the synaptic weights further comprises sensing the synaptic weights in parallel from the NAND word lines and performing a plurality of the neural network computing operations in parallel using the synaptic weights.
16. The method of claim 11 , wherein the method further comprises:
receiving a physical block identifier from a device that is external to the memory array, that identifies a source block within the memory array that corresponds to the single-bit-per-cell NVM NAND elements;
reading the synaptic weights from the single-bit-per-cell elements of the source block of the memory array identified by the physical block identifier received from the external device;
updating the synaptic weights of the source block using the neural network computing component that performs the neural network computing operation;
folding updated synaptic weights into multiple-bit-per-cell NVM NAND elements of a target block of the memory array that corresponds to an updated physical block identifier; and
sending a command completion response to the external device that notifies the external device that the neural network computing operation has completed and provides the updated physical block identifier of one or more synaptic weights that have been changed by the on-chip copy with update component.
17. The method of claim 11 ,
wherein the NVM NAND elements store the updated synaptic weights within a plurality of word lines;
wherein the neural network computing component is one of a plurality of neural network computing components formed on or in the least one memory die of the memory array; and
wherein the plurality of neural network computing components access the updated synaptic weights in parallel from the plurality of word lines and perform the neural network computing operations in parallel using the updated synaptic weight values.
18. An apparatus for use with a memory array of memory dies that include non-volatile memory (NVM) NAND elements, the apparatus comprising:
means for sensing synaptic weights of a neural network stored within the NVM NAND elements;
means for performing a read-modify-write operation comprising obtaining neural network training data from a memory external to the memory array, updating the synaptic weights based on the training data using neural network computing circuits formed on or in at least one memory die of the memory array, and storing the updated synaptic weights in the single-bit-per cell elements of the memory array; and
means for performing an on-chip copy and update within the memory array by performing a fold operation in which at least some of the updated synaptic weights are sensed from single-bit-per-cell NVM NAND elements of the memory array, are further updated using the neural network computing circuits, and are folded into multiple-bit-per-cell NVM NAND elements of the memory die.
19. The apparatus of claim 18 , wherein the means for performing the on-chip copy and update further comprises:
means for receiving a physical block identifier from a device that is external to the memory array, that identifies a source block within the memory array that corresponds to the single-bit-per-cell NVM NAND elements;
means for reading the synaptic weights from the single-bit-per-cell NVM NAND elements of the source block of the memory array identified by the physical block identifier received from the external device;
means for updating the synaptic weights of the source block using the means for performing the neural network computation;
means for folding updated synaptic weights into multiple-bit-per-cell NVM NAND elements of a target block of the memory array that corresponds to an updated physical block identifier; and
means for sending a command completion response to the external device that notifies the external device that the neural network computing operation has completed and provides the updated physical block identifier of one or more synaptic weights that have been changed by the on-chip copy with update component.