Systems and methods for use with recurrent neural networks
Recurrent Neural Networks (RNNs) wherein a non-volatile memory (NVM) array provides a memory bank for the RNN. The RNN may be a Neural Turing Machine (NTM) and the memory bank may be an NTM matrix stored in the NVM array. In some examples, an NTM controller sets the size of the NTM matrix based on a storage access granularity of the NVM array. For instance, if the NVM reads and writes data in flash memory unit (FMUs), the NTM controller sets the size of the NTM matrix to correspond to the size of an integer number of FMUs. In some examples, the NVM array includes on-chip NTM circuitry configured to perform at least some NTM read head and write head operations. Threshold-based processing is described that can reduce an amount of NTM data read from the NVM array. In other examples, volatile memory is employed rather than an NVM array.
1. A data storage device, comprising:
a non-volatile memory (NVM) array configured to store Neural Turing Machine (NTM) data, the NVM array having a storage access granularity, wherein the NVM array includes a die with on-chip circuitry configured to perform NTM read vector operations and NTM write vector operations on the NTM data; and
a controller coupled to the NVM array, the controller comprising one or more processors configured, individually or in combination, to:
set a size of an NTM matrix for storage within the NVM array based on the storage access granularity of the NVM array;
store the NTM data within the NTM matrix of the NVM array; and
determine at least one NTM output value using the NTM data stored within the NTM matrix of the NVM array by being further configured to control the on-chip circuitry of the die of the NVM array to perform an NTM write vector operation by controlling the on-chip circuitry to:
receive a set of NTM weight values from the controller;
count or sum weight values in the set of NTM weight values that exceed a first non-zero threshold; and
selectively update all or a portion of the NTM matrix within the NVM array based on whether the count or the sum exceeds a second non-zero threshold.
2. The data storage device of claim 1 , wherein the storage access granularity of the NVM array is based on a memory unit comprising a flash memory unit (FMU), a word-line, a block, and a meta-block.
3. The data storage device of claim 1 , wherein the storage access granularity of the NVM array comprises a flash memory unit (FMU), and wherein the processor is configured to set the size of the NTM memory bank based on a size of an integer number of the FMUs.
4. The data storage device of claim 3 , wherein the one or more processors are further configured, individually or in combination, to set the size of the NTM memory bank equal to the size of the integer number of the FMUs.
5. The data storage device of claim 3 , wherein the one or more processors are further configured, individually or in combination, to set the size of the NTM memory bank to be less than the size of the integer number of the FMUs and greater than the size of the integer number minus one of the FMUs.
6. The data storage device of claim 1 , wherein the NTM memory bank is a matrix of size N×D, and wherein the one or more processors are further configured, individually or in combination, to set one or both of N and D based on the storage access granularity of the NVM array.
7. The data storage device of claim 1 , wherein the one or more processors are further configured, individually or in combination, to:
generate an NTM weight value for applying to a corresponding one of a set of NTM matrix values stored within the NVM array;
compare the generated weight value to the first non-zero threshold;
retrieve the corresponding matrix value from the NVM array, in response to a determination that the weight value exceeds the first non-zero threshold, and apply the weight value to the corresponding matrix value to determine a corresponding NTM read value; and
set the corresponding NTM read value to zero or a constant value, in response to a determination that the weight value does not exceed the first non-zero threshold, without retrieving the corresponding matrix value from the NVM array.
8. The data storage device of claim 1 , wherein the on-chip circuitry is further controlled to:
receive the first threshold and NTM weight values from the controller for applying to corresponding NTM matrix values stored within the NVM array;
compare the received weight values to the first threshold to identify individual weight values that exceed the threshold;
sequentially read NTM matrix values from the NVM array that correspond to the identified weight values;
multiply the sequentially read NTM matrix values with the corresponding identified weight values;
sum the results of the multiplication of the sequentially read NTM matrix values with the corresponding identified weight values; and
output the summed results to the controller.
9. The data storage device of claim 1 , wherein the on-chip circuitry is controlled to selectively update all or a portion of the NTM matrix by being further controlled to:
invalidate, in response to a determination that the count or the sum exceeds the second non-zero threshold, a first set of memory array locations that store the NTM matrix and compute and write an updated NTM matrix to a second set of memory locations within the NVM array; and
identify, in response to a determination that the count or the sum does not exceed the second non-zero threshold, portions of the updated NTM matrix that have corresponding weights that exceed the first threshold and write the portions of the updated NTM matrix to the second memory locations within the NVM array.
10. The data storage device of claim 1 , wherein the on-chip circuitry of the NVM array is further controlled to update an NVM management table stored in the NVM array to identify the second memory locations of the NVM array.
11. The data storage device of claim 1 , wherein the NVM array comprises a NAND NVM array.
12. The data storage device of claim 1 , wherein the one or more processors are further configured, individually or in combination, to train a recurrent neural network by being further configured to control the on-chip circuitry of the die of the NVM array to perform the NTM read vector operations and the NTM write vector operations on attentionally-selected portions of the NTM matrix.
13. A method for use by a controller coupled to a non-volatile memory (NVM) array, wherein the NVM array includes a die with on-chip circuitry configured to perform Neural Turing Machine (NTM) read vector operations and NTM write vector operations on NTM data, the method comprising:
setting a size of an NTM matrix for storage within the NVM array based on a storage access granularity of the NVM array;
storing the NTM data within the NTM matrix of the NVM array; and
determining at least one NTM output value using the NTM data stored within the NTM matrix of the NVM array by controlling the on-chip circuitry of the die of the NVM array to perform at least one NTM write vector operation by:
receiving a set of NTM weight values from the controller;
counting or summing weight values in the set of NTM weight values that exceed a first non-zero threshold; and
selectively updating all or a portion of the NTM matrix within the NVM array based on whether the count or the sum exceeds a second non-zero threshold.
14. The method of claim 13 , wherein the storage access granularity of the NVM array is based on a memory unit comprising a flash memory unit (FMU), a word-line, a block, and a meta-block.
15. The method of claim 13 , wherein the storage access granularity of the NVM array comprises a flash memory unit (FMU), and wherein the size of the NTM memory bank is set based on a size of an integer number of the FMUs.
16. The method of claim 15 , wherein the size of the NTM memory bank is set equal to the size of the integer number of the FMUs.
17. The method of claim 16 , wherein the size of the NTM memory bank is set to be less the size of the integer number of the FMUs and greater than the size of the integer number minus one of the FMUs.
18. The method of claim 13 , wherein the NTM memory bank is a matrix of size N×D, and one or both of N and D are set based on the storage access granularity of the NVM array.
19. The method of claim 13 , wherein the method further comprises:
generating an NTM weight value for applying to a corresponding one of a set of NTM matrix values stored within the NVM array;
comparing the generated weight value to the first non-zero threshold;
retrieving the corresponding matrix value from the NVM array, in response to a determination that the weight value exceeds the first non-zero threshold, and applying the weight value to the corresponding matrix value to determine a corresponding NTM read value; and
setting the corresponding NTM read value to zero or a constant value, in response to a determination that the weight value does not exceed the first non-zero threshold, without retrieving the corresponding matrix value from the NVM array.
20. The method of claim 13 , wherein the NVM array comprises a NAND NVM array.
21. The method of claim 13 , wherein selectively updating all or a portion of the NTM matrix within the NVM array comprises:
invalidating, in response to a determination that the count or the sum exceeds the second non-zero threshold, a first set of memory array locations that store the NTM matrix and then computing an updated NTM matrix using the on-chip circuitry and writing the updated NTM matrix to a second set of memory locations within the NVM array; and
identifying, in response to a determination that the count or the sum does not exceed the second zero threshold, portions of the updated NTM matrix that have corresponding weights that exceed the first threshold and writing the portions of the updated NTM matrix to the second memory locations within the NVM array.
22. A method for use by on-chip circuitry of a non-volatile memory (NVM) array, wherein the NVM array includes a die with on-chip circuitry configured to perform Neural Turing Machine (NTM) write vector operations on NTM data, the method comprising performing an NTM write vector operation using the on-chip circuitry of the die by:
receiving a set of NTM weight values from a controller for applying to a corresponding NTM matrix stored within the NVM array;
identifying weight values in the set of weight values that exceed a first non-zero threshold and counting or summing the identified weight values;
invalidating, in response to a determination that the count or a sum exceeds a second non-zero threshold, a first set of memory array locations that store the NTM matrix and computing an updated NTM matrix using the on-chip circuitry of the die and writing the updated NTM matrix to a second set of memory locations within the NVM array; and
identifying, in response to a determination that the count or sum does not exceed the second non-zero threshold, portions of an updated NTM matrix that have corresponding weights that exceed the first non-zero threshold and writing the portions of the updated NTM matrix to second memory locations within the NVM array.
23. The method of claim 22 , further comprising updating an NVM management table stored in the NVM array to identify the second memory locations of the NVM array.
24. A non-volatile memory (NVM) die, comprising:
NVM data storage blocks configured to store Neural Turing Machine (NTM) data comprising an NTM matrix;
on-chip NTM write vector circuitry on the die configured to:
receive a set of NTM weight values from a controller for applying to the NTM matrix stored within the data storage blocks of the NVM die;
identify weight values in the set of weight values that exceed a first non-zero threshold;
count or sum the identified weight values;
invalidate, in response to a determination that the count or a sum exceeds a second non-zero threshold, a first set of memory locations within the data storage blocks that store the NTM matrix and compute and write an updated NTM matrix to a second set of memory locations within the data storage blocks; and
identify, in response to a determination that the count or sum does not exceed the second non-zero threshold, portions of the updated NTM matrix that have corresponding weights that exceed the first non-zero threshold and write the portions of the updated NTM matrix to the second memory locations within the NVM array.