Machine learning for temperature compensation
A method of temperature compensation to read a flash memory device includes determining a state of the flash memory device. An action is selected with a maximum Q-value from a Q-table for the current state during exploitation. A read operation of a code word from the flash memory device is conducted using one or more parameters according to the selected action. The code word is decoded with an error correction code (ECC) process.
1. A method of temperature compensation to read a flash memory device, comprising:
determining a current environment state of the flash memory device;
selecting an action with a maximum Q-value from a Q-table for the current environment state during exploitation, the Q-table comprising a plurality of actions for each of a plurality of environment states of the flash memory device;
conducting a read operation of a code word from the flash memory device using one or more parameters according to the selected action; and
decoding the code word with an error correction code (ECC) process.
2. The method of claim 1 , further comprising selecting an action with a non-maximum Q-value using a probability of p=1−ε from the Q-table during exploration.
3. The method of claim 1 , further comprising:
determining an estimated bit error rate (BER) of the code word from the read operation, wherein the decoding the code word with the ECC process occurs if the estimated BER is at or below a decoding limit; and
selecting another action from the Q-table for the current environment state to conducted another read operation of the code word from the flash memory device using one or more parameters according to the selected another action.
4. The method of claim 1 , wherein determining the current environment state of the flash memory device is a previous read access environment state in a sequential or a random read operation of the flash memory device, wherein the selecting the action determines a next environment state of the flash memory device.
5. The method of claim 1 , wherein a triggering event selected from a group consisting of a temperature difference between a read temperature and a program temperature of a stored data is higher than a threshold, a decoding failure of an ECC engine, a host request, and a recovery mode by a storage device controller is performed before the selecting the action.
6. The method of claim 1 , wherein the environment state of the flash memory device comprises one or more characteristics selected from a group consisting of a current environment state temperature compensation (TC) parameters, a previous environment state TC parameters, a current environment state syndrome weight, a previous environment state syndrome weight, a time after programming, a programming temperature, a read temperature, a number of program/erase cycles, and quantizations thereof.
7. The method of claim 1 , wherein the one or more parameters of each of the plurality of actions are selected from a group consisting of a bit line voltage, a sense amplification time, and read threshold voltages.
8. The method of claim 1 , wherein the plurality of actions comprises a first set of actions comprising a first set of one or more parameters and comprises a second set of actions comprising a second set of one or more parameters, the first set of one or more parameters determined by offline characterization of the flash memory device and the second set of one or more parameters determined by online characterization of the flash memory device.
9. The method of claim 1 , wherein the Q-table comprises a Q-value for each of the plurality of actions which is updated based upon a reward to minimize accumulative decoding latency.
10. A data storage device, comprising:
a memory die;
a control die coupled to the memory die, the control die including Q-learning logic comprising a bit error rate (BER) estimation module and a Q-table; and
a storage device controller coupled to the control die;
the Q-learning logic operable to:
determine a current environment state of the memory die;
select an action with a maximum Q-value from a Q-table for the current environment state during exploitation, wherein the Q-table comprises a plurality of actions for each of a plurality of environment states of the memory die; and
conduct a read operation of a code word from the memory die using one or more parameters according to the selected action.
11. The data storage device of claim 10 , wherein the control die further comprises an error correction code (ECC) engine, wherein the Q-learning logic is further operable to transfer the code word to the ECC engine for decoding.
12. The data storage device of claim 10 , wherein the storage device controller further comprises an error correction code (ECC) engine, wherein the Q-learning logic is further operable to transfer the code word to the ECC engine for decoding.
13. The data storage device of claim 10 , wherein the determined current environment state of the memory die is a previous read access environment state in a sequential or a random read operation of the memory die, wherein the selected action determines a next environment state of the memory die.
14. The data storage device of claim 10 , wherein the Q-learning logic is for a current read operation after a triggering event selected from a group consisting of a temperature difference between a read temperature and a program temperature of a stored data is higher than a threshold, a decoding failure of an ECC engine, a host request, or a recovery mode by a storage device controller.
15. The data storage device of claim 10 , wherein the environment state of the memory die comprises one or more characteristics selected from a group consisting of a current environment state temperature compensation (TC) parameters, a previous environment state TC parameters, a current environment state syndrome weight, a previous environment state syndrome weight, a time after programming, a programming temperature, a read temperature, a number of program/erase cycles, and quantizations thereof.
16. The data storage device of claim 10 , wherein the one or more parameters of each of the plurality of actions are selected from a group consisting of a bit line voltage, a sense amplification time, and read threshold voltages.
17. The data storage device of claim 10 , wherein the plurality of actions comprises a first set of actions comprising a first set of one or more parameters and comprises a second set of actions comprising a second set of one or more parameters, the first set of one or more parameters determined by offline characterization of the memory die and the second set of one or more parameters determined by online characterization of the memory die.
18. The data storage device of claim 10 , wherein the Q-table comprises a Q-value for each of the plurality of actions which is updated based upon a reward to minimize accumulative decoding latency.
19. A data storage device, comprising:
a memory die; and
a controller coupled to the memory die, the controller comprising:
a bit error rate (BER) estimation module;
a Q-table; and
an error correction code (ECC) engine;
the controller operable to:
determine a current environment state of the memory die;
select an action with a maximum Q-value from a Q-table for the current environment state during exploitation;
conduct a read operation of a code word from the memory die using one or more parameters according to the selected action; and
decode the code word with the (ECC) engine.
20. The data storage device of claim 19 , wherein the Q-table comprises a Q-value for each of a plurality of actions and is updated based upon a reward to minimize accumlative decoding latency.