Hybrid memory management of non-volatile memory (NVM) devices for use with recurrent neural networks
Recurrent Neural Networks (RNNs) wherein a non-volatile memory (NVM) array provides a memory bank for the RNN. The RNN may include a Neural Turning Machine (NTM) and the memory bank may be an NTM matrix stored in the NVM array. In some examples, a data storage device (DSD) that controls the NVM array includes both a data storage controller and a separate NTM controller. The separate NTM controller accesses the NTM matrix of the NVM array directly while bypassing flash translation layer (FTL) components of the data storage controller. Additionally, various majority wins error detection and correction procedures are described, as well as various disparity count-based procedures.
1. A data storage device, comprising:
a non-volatile memory (NVM) array; and
a controller coupled to the NVM array and having a processor configured to:
receive data for storage in the NVM array;
determine whether the data is neural network data or non-neural network data;
in response to a determination the data is neural network data,
store at least three copies of the neural network data in the NVM array;
read the at least three copies of the neural network data from the NVM array; and
perform a majority rules error correction procedure on the at least three copies of the neural network data read from the NVM array; and
in response to a determination the data is non-neural network data,
store the non-neural network data in the NVM array;
read the non-neural network data from the NVM array; and
perform an error correction code (ECC) procedure on the non-neural network data, the ECC procedure being different from the majority rules error correction procedure.
2. The data storage device of claim 1 , wherein the neural network data corresponds to a neural network that is configured so that an uncorrected error in the neural network data increases regularization of the neural network.
3. The data storage device of claim 1 , wherein the processor is further configured to:
scramble the neural network data before storing at least three copies of the scrambled neural network data in the NVM array;
perform the majority rules error correction procedure on the at least three scrambled copies of the neural network data read from the NVM array; and
unscramble the scrambled neural network data.
4. The data storage device of claim 1 , wherein the processor is further configured to perform the majority rules error correction procedure on the neural network data by being further configured to:
detect a difference between the at least three copies of the neural network data read from the NVM array; and
identify a correct version of the neural network data based on a majority rules policy.
5. The data storage device of claim 4 , wherein the processor is further configured to re-write the correct version of the neural network data in a new memory location within the NVM array.
6. The data storage device of claim 4 , wherein the processor is further configured to re-write the correct version of the neural network data by being further configured to erase the memory location of an incorrect copy of the data and writing the correct version of the data to the memory location that had been erased.
7. The data storage device of claim 1 , wherein the at least three copies consist of an odd number of copies.
8. The data storage device of claim 1 , wherein the processor is further configured to store the at least three copies with at least one of the copies stored on a different die within the NVM array from the other copies.
9. The data storage device of claim 1 , wherein the processor is further configured to perform a wear leveling procedure on the NVM array by being further configured to:
track a program rate of blocks of the NVM array that store the neural network data to identify a most-used block and a least-used block; and
periodically swap neural network data stored in the most-used block with neural network data stored in the least-used block.
10. The data storage device of claim 1 , wherein the neural network data corresponds to a recurrent neural network (RNN) comprising at least one of a Memory Augmented Neural Network (MANN) or a Neural Turing Machine (NTM) network.
11. The data storage device of claim 1 , wherein the controller comprises a data storage controller and a separate recurrent neural network (RNN) controller.
12. The data storage device of claim 11 ,
wherein the data storage controller is configured to access at least a first portion of the NVM array that has physical memory addresses allocated to the data storage controller; and
wherein the RNN controller is configured to access a second portion of the NVM array, different from the first portion, that has physical memory addresses allocated to the RNN controller, wherein the second portion of the NVM array is configured for use as an RNN memory bank.
13. A method for use with a data storage device that includes a non-volatile memory (NVM) array, the method comprising:
receiving data for storage in the NVM array;
determining whether the data is neural network data or non-neural network data;
in response to a determination the data is neural network data,
storing at least three copies of the neural network data in the NVM array;
reading the at least three copies of the neural network data from the NVM array; and
performing a majority rules error correction procedure on the at least three copies of the neural network data read from the NVM array; and
in response to a determination the data is non-neural network data,
storing the non-neural network data in the NVM array;
reading the non-neural network data from the NVM array; and
performing an error correction code (ECC) procedure on the non-neural network data, the ECC procedure being different from the majority rules error correction procedure.
14. The method of claim 13 , wherein a likelihood of a bit error coinciding on a same bit at two separate storage locations of the stored neural network data is below a threshold representative of an acceptable fault tolerance.
15. The method of claim 13 , wherein the neural network data corresponds to a neural network that is configured so that an uncorrected error in the neural network data increases regularization of the neural network.
16. The method of claim 13 , wherein the majority rules error correction procedure comprises:
detecting a difference between the at least three copies of the neural network data read from the NVM array; and
identifying a correct version of the neural network data based on a majority rules policy.
17. The method of claim 16 , further comprising re-writing the correct version of the neural network data in a new memory location within the NVM array.
18. The method of claim 16 , further comprising re-writing the correct version of the neural network data by erasing a memory location of an incorrect copy of the data and writing the correct version of the data to the memory location that had been erased.
19. The method of claim 13 , further comprising:
scrambling the neural network data before storing at least three copies of the scrambled neural network data in the NVM array;
performing the majority rules error correction procedure on the at least three scrambled copies of the neural network data read from the NVM array; and
unscrambling the scrambled neural network data.
20. The method of claim 13 , wherein the at least three copies consist of an odd number of copies.
21. The method of claim 13 , further comprising storing the at least three copies with at least one of the copies stored on a different die within the NVM array from the other copies.
22. The method of claim 13 , further comprising performing a wear leveling procedure on the NVM array by:
tracking a program rate of blocks of the NVM array that store the neural network data to identify a most-used block and a least-used block; and
periodically swapping neural network data stored in the most-used block with neural network data stored in the least-used block.
23. An apparatus for use with a data storage device that includes a non-volatile memory (NVM) array, the apparatus comprising:
means for receiving data for storage in the NVM array;
means for determining whether the data is neural network data or non-neural network data;
means, operative in response to a determination the data is neural network data, for storing at least three copies of the neural network data in the NVM array;
means for reading the at least three copies of the neural network data from the NVM array;
means for performing a majority rules error correction procedure on the at least three copies of the neural network data read from the NVM array;
means, operative in response to a determination the data is non-neural network data, for storing the non-neural network data in the NVM array;
means for reading the non-neural network data from the NVM array; and
means for performing an error correction code (ECC) procedure on the non-neural network data, the ECC procedure being different from the majority rules error correction procedure.
24. A non-transitory computer-readable medium comprising one or more instructions which when executed by a controller of a data storage device having a non-volatile memory (NVM) array causes the controller to:
receive data for storage in the NVM array;
determine whether the data is neural network data or non-neural network data;
in response to a determination the data is neural network data, store at least three copies of the neural network data in the NVM array, read the at least three copies of the neural network data from the NVM array, and perform a majority rules error correction procedure on the at least three copies of the neural network data read from the NVM array; and
in response to a determination the data is non-neural network data, store the non-neural network data in the NVM array, read the non-neural network data from the NVM array, and perform an error correction code (ECC) procedure on the non-neural network data, the ECC procedure being different from the majority rules error correction procedure.