Non-volatile memory die with deep learning neural network
Exemplary methods and apparatus are provided for implementing a deep learning accelerator (DLA) or other neural network components within the die of a non-volatile memory (NVM) apparatus using, for example, under-the-array circuit components within the die. Some aspects disclosed herein relate to configuring the under-the-array components to implement feedforward DLA operations. Other aspects relate to backpropagation operations. Still other aspects relate to using an NAND-based on-chip copy with update function to facilitate updating synaptic weights of a neural network stored on a die. Other aspects disclosed herein relate to configuring a solid state device (SSD) controller for use with the NVM. In some aspects, the SSD controller includes flash translation layer (FTL) tables configured specifically for use with neural network data stored in the NVM.
1. An apparatus, comprising:
a non-volatile memory (NVM) array comprising a memory die, the NVM array configured to perform an on-chip neural network learning operation that comprises reading a weight unit from one or more source blocks within the NVM array, modifying the weight unit in accordance with the neural network learning operation, and writing the modified weight unit to a target block in the NVM array; and
a processor coupled to the NVM array and configured to
generate a first mapping table that stores initial weight units and corresponding virtual block identifiers and maps the initial weight units to corresponding virtual locations within the NVM array via the virtual block identifiers, wherein the initial weight units correspond to initial synaptic weights of a neural network,
generate a second mapping table that stores the virtual block identifiers and corresponding initial physical block identifiers and maps the virtual block identifiers to corresponding physical locations within the NVM array via the initial physical block identifiers,
send, to the NVM array, a particular physical block identifier that corresponds to the one or more source blocks, wherein the particular physical block identifier is obtained from the second mapping table,
receive a command completion response from the NVM array that notifies the processor that the on-chip neural network learning operation has completed and provides updated physical block identifiers of one or more weight units that have been changed by the on-chip neural network learning operation at the NVM array, and
update the initial physical block identifiers within the second mapping table that correspond to the one or more weight units that have been changed within the NVM array, wherein the initial physical block identifiers are updated with the updated physical block identifiers.
2. The apparatus of claim 1 , wherein the NVM array comprises NAND flash elements.
3. The apparatus of claim 1 ,
wherein the on-chip neural network learning operation comprises a backpropagation operation that changes the one or more initial weight units, and
wherein the command completion response identifies the updated physical block identifiers for the one or more weight units that have been changed during the backpropagation operation.
4. The apparatus of claim 3 ,
wherein the on-chip neural network learning operation further comprises a feedforward operation, and
wherein the processor is further configured to send the updated physical block identifiers to the NVM array to perform the feedforward operation on the one or more weight units that were changed during the backpropagation operation.
5. The apparatus of claim 1 ,
wherein the NVM array comprises single-bit-per-cell blocks and multiple-bit-per-cell blocks, and wherein each single-bit-per-cell block corresponds to a single one of the virtual block identifiers and each multiple-bit-per-cell block corresponds to two or more of the virtual block identifiers,
wherein the processor is further configured to generate the second mapping table by mapping each virtual block identifier either to a corresponding single-bit-per-cell block or to a portion of a corresponding multiple-bit-per-cell block, and
wherein the source block is one of the single-bit-per-cell blocks and wherein the target block includes at least one of the multiple-bit-per-cell blocks.
6. The apparatus of claim 5 ,
wherein the NVM memory array is further configured to perform a fold operation in which the one or more initial weight units are read from a plurality of the single-bit-per-cell blocks, are changed in accordance with an on-chip backpropagation procedure, and are folded into at least one of the multiple-bit-per-cell blocks, and
wherein the processor is further configured to apply the updated physical block identifiers received from the NVM array to the second mapping table to update the second mapping table to match the fold operation without updating corresponding values within the first mapping table.
7. The apparatus of claim 1 , wherein the processor is further configured to allocate the target block in the NVM array for use by the on-chip neural network learning operation and to then release the one or more source blocks after the command completion response is received.
8. A method for use by a controller of an apparatus that includes a memory array of non-volatile memory (NVM) elements, the method comprising:
generating a first mapping table that stores initial weight units and corresponding virtual block identifiers and maps the initial weight units to corresponding virtual locations within the memory array via the virtual block identifiers, wherein the initial weight units correspond to initial synaptic weights of a neural network;
generating a second mapping table that stores the virtual block identifiers and corresponding initial physical block identifiers and maps the virtual block identifiers to corresponding physical locations within the memory array via the initial physical block identifiers;
converting a particular initial weight unit to a particular virtual block identifier using the first table;
converting the particular virtual block identifier to a particular initial physical block identifier using the second table;
sending the particular initial physical block identifier to the memory array for processing by an on-chip neural network learning operation within the memory array that comprises reading the initial weight unit from one or more source blocks within the memory array, modifying the initial weight unit in accordance with the neural network learning operation, and writing the modified weight unit to a target block in the memory array;
receiving a command completion response from the memory array that notifies the controller that the on-chip neural network learning operation has completed and provides updated physical block identifiers of one or more weight units that have been changed by the on-chip neural network learning operation at the memory array; and
updating the initial physical block identifiers within the second mapping table that correspond to the one or more weight units that have been changed by the memory array wherein the initial physical block identifiers are updated with the updated physical block identifiers.
9. The method of claim 8 ,
wherein the on-chip neural network learning operation comprises a backpropagation operation that changes the one or more initial weight units, and
wherein the command completion response identifies the updated physical block identifiers for the one or more initial weight units changed during the backpropagation operation.
10. The method of claim 9 ,
wherein the on-chip neural network learning operation further comprises a feedforward operation, and
wherein the method further comprises sending the updated physical block identifiers to the memory array to perform the feedforward operation on the one or more weight units that were changed during the backpropagation operation.
11. The method of claim 8 ,
wherein the memory array comprises single-bit-per-cell blocks and multiple-bit-per-cell blocks, and wherein each single-bit-per-cell block corresponds to a single one of the virtual block identifiers and each multiple-bit-per-cell block corresponds to two or more of the virtual block identifiers,
wherein the method further comprises generating the second mapping table by mapping each virtual block identifier either to a corresponding single-bit-per-cell block or to a portion of a corresponding multiple-bit-per-cell block, and
wherein the source block is one of the single-bit-per-cell blocks and wherein the target block includes at least one of the multiple-bit-per-cell blocks.
12. The method of claim 11 ,
wherein the memory array is configured to perform a fold operation in which the one or more initial weight units are read from a plurality of the single-bit-per-cell blocks, are updated in accordance with an on-chip backpropagation procedure, and are folded into at least one of the multiple-bit-per-cell blocks, and
wherein the method further comprises applying the updated physical block identifiers received from the memory array to the second mapping table to update the second mapping table to match the fold operation without updating corresponding values within the first mapping table.
13. The method of claim 8 , further comprising allocating the target block in the memory array for use by the on-chip neural network learning operation and then releasing the one or more source blocks after the command completion response is received.
14. An apparatus for use with a non-volatile memory (NVM) array, the apparatus comprising:
means for generating a first mapping table that stores initial weight units and corresponding virtual block identifiers and maps the initial weight units to corresponding virtual locations within the memory array via the virtual block identifiers, wherein the initial weight units correspond to initial synaptic weights of a neural network;
means for generating a second mapping table that stores the virtual block identifiers and corresponding initial physical block identifiers and maps the virtual block identifiers to corresponding physical locations within the memory array via the initial physical block identifiers;
means for converting a particular initial weight unit to a particular virtual block identifier using the first table;
means for converting the particular virtual block identifier to a particular initial physical block identifier using the second table;
means for sending the particular initial physical block identifier to a memory die of the NVM array for processing by an on-chip neural network learning operation that reads the initial weight unit from one or more source blocks within the NVM array, modifies the initial weight unit in accordance with the neural network learning operation, and writes the modified weight unit to a target block in the NVM array;
means for receiving a command completion response from the NVM array indicating that the on-chip neural network learning operation has completed, wherein the command completion response provides updated physical block identifiers of one or more weight units that have been changed by the on-chip neural network learning operation at the NVM array; and
means for updating the initial physical block identifiers within the second mapping table of the one or more weight units that have been changed, wherein the initial physical block identifiers are updated with the updated physical block identifiers.
15. The apparatus of claim 14 ,
wherein the on-chip neural network learning operation comprises a backpropagation operation that changes the one or more initial weight units, and
wherein the command completion response identifies the updated physical block identifiers for the one or more initial weight units changed during the backpropagation operation.
16. The apparatus of claim 15 ,
wherein the on-chip neural network learning operation further comprises a feedforward operation, and
wherein the apparatus further comprises means for sending the updated physical block identifiers to the NVM array to perform the feedforward operation on the one or more weight units that were changed during the backpropagation operation.
17. The apparatus of claim 14 , further comprising means for allocating the target block in the NVM array for use by the on-chip neural network learning operation and means for releasing the one or more source blocks after the command completion response is received.
18. An apparatus, comprising:
a non-volatile memory (NVM) array comprising a memory die, the NVM array configured to perform an on-chip neural network learning operation that reads a weight unit from one or more source blocks within the NVM array, modifies the weight unit in accordance with the neural network learning operation, and writes the modified weight unit to a target block in the NVM array;
an allocation component configured to allocate the target block in the NVM array for the one or more source blocks for use by the on-chip neural network learning operation;
an output component configured to send, to the NVM array, a particular physical block identifier that corresponds to the one or more source blocks;
a receive component configured to receive a command completion response indicating that the on-chip neural network learning operation has completed, wherein the command completion response provides the updated physical block identifiers of one or more weight units that have been changed by the on-chip neural network learning operation at the NVM array;
a release component configured to release the one or more source blocks; and
an update component configured to update physical block identifiers within a mapping table that correspond to the one or more weight units that have been changed within the NVM array, wherein the physical block identifiers are updated with the updated physical block identifiers.
19. The apparatus of claim 18 , wherein the NVM array comprises NAND flash elements.
20. The apparatus of claim 18 , wherein the one or more weight units include one or more initial synaptic weight values.