Parameterized iterative message passing decoder
View Patent ↗Technology is described herein for learning parameters for a parameterized iterative message passing decoder, and to a corresponding parameterized iterative message passing decoder. Learning the parameters may adapt the decoder to statistical dependencies introduced by the specific code's graph. Taking into account the statistical dependencies may allow the code to be shorter and/or denser. Note that the statistical dependencies in the graph may be extremely complex. Machine learning may be used to learn the parameters. The parameters may be learned when decoding data stored in the memory device. Learning the parameters may adapt the decoder to properties of data stored in the memory device, physical properties of the memory device, and/or patterns in host data.
1. An apparatus, comprising:
non-volatile memory;
a parameterized iterative message passing decoder comprising:
a message passing computer configured to perform message passing computations that comprise a parameter;
a router configured to pass noisy versions of codewords through the decoder to determine mutual information between a decoded result for each noisy codeword and clean versions of the codewords;
a parameter learner configured to learn the parameter based on a rate of change of the mutual information with respect to the parameter; and
a control circuit configured to access encoded data from the non-volatile memory and provide the encoded data to the decoder, wherein the decoder is configured to decode the encoded data using the learned parameter.
2. The apparatus of claim 1 wherein:
the decoder further comprises a gradient computer configured to determine the rate of change of the mutual information with respect to the parameter for each noisy codeword.
3. The apparatus of claim 1 , wherein the control circuit is configured to pass batches of the noisy versions of the codewords to the decoder, wherein the parameter learner is configured to revise the parameter with each batch.
4. The apparatus of claim 3 , wherein the control circuit is configured to pass additional batches of the noisy versions of the codewords to the decoder until the mutual information saturates.
5. The apparatus of claim 1 , wherein to the parameter learner is configured to attempt to minimize a bit error rate that results from passing the noisy codewords through the decoder.
6. The apparatus of claim 1 , wherein the noisy codewords are log likelihood ratios based on data that was previously stored in the non-volatile memory.
7. The apparatus of claim 6 , wherein the parameter learner is further configured to adapt the decoder to noise characteristics of the non-volatile memory that change over time, the parameter learner is configured to adapt the decoder based on results of passing the noisy codewords through the decoder.
8. The apparatus of claim 6 , wherein the parameter learner is further configured to adapt the decoder to host traffic characteristics, based on parameters that are learned from results of passing the noisy codewords through the decoder.
9. The apparatus of claim 1 , wherein the noisy codewords are based on cold data that was previously stored in the non-volatile memory, wherein the parameter learner is further configured to adapt the decoder to specific noise realizations of the cold data.
10. The apparatus of claim 1 , wherein the decoder further comprises storage that describes a sparse bipartite graph having nodes and edges, wherein the parameter learner is configured to learn the parameter for ones of the edges.
11. The apparatus of claim 1 , wherein the mutual information is based on a posteriori log likelihood ratios (LLRs) at an output of the decoder and the clean codewords.
12. A machine-implemented method comprising:
passing a batch of noisy versions of codewords through a parameterized iterative message passing decoder, including performing message passing computations comprising parameters;
revising the parameters based on mutual information between a-posteriori log likelihood ratios (LLRs) of passing the batch of noisy codewords through the parameterized iterative message passing decoder and clean versions of the codewords, including revising the parameters based on how much the mutual information would change with respect to a change in the parameters for each noisy codeword;
repeating the passing and the revising with additional batches of noisy versions of codewords until the parameters are learned;
accessing encoded data that was stored in non-volatile memory; and
decoding the encoded data using the revised parameters in the parameterized iterative message passing decoder.
13. The machine-implemented method of claim 12 , wherein repeating the passing and the revising with additional batches of noisy version of codewords until the parameters are learned comprises:
attempting to maximize the mutual information between the a-posteriori LLRs and the clean codewords.
14. The machine-implemented method of claim 12 , further comprising:
encoding data in accordance with a code used by the iterative message passing decoder;
storing the encoded data in a non-volatile storage memory array; and
reading back the encoded data from the non-volatile storage memory array;
wherein the noisy codewords are based on the encoded data read back from the non-volatile storage memory array.
15. The machine-implemented method of claim 12 , wherein the message passing computations comprise belief propagation rules.
16. The machine-implemented method of claim 12 , wherein;
the parameterized iterative message passing decoder is based on a sparse bipartite graph having cycles; and
revising learning the parameters comprises: learning statistical dependencies between the message passing computations.
17. The machine-implemented method of claim 16 , wherein the message passing computations comprise messages between bit nodes and check nodes in the sparse bipartite graph.
18. A non-volatile storage device, comprising:
non-volatile storage cells;
means for performing message passing computations on a batch of noisy versions of codewords over one or more iterations of a sparse bipartite graph to generate a-posteriori log-likelihood ratios (LLRs), wherein ones of the message passing computations have one or more parameters, wherein the message passing computations comprise messages between bit nodes and check nodes in the sparse bipartite graph;
means for revising the one or more parameters based on mutual information between the a-posteriori LLRs and clean versions of the codewords, including means for revising the one or more parameters based on a gradient of ones of the parameters with respect to the mutual information;
means for revising the one or more parameters based on performing the message passing computations on additional batches of noisy versions of codewords over one or more iterations of the sparse bipartite graph to generate a-posteriori log-likelihood ratios (LLRs) and revising the one or more parameters based on the mutual information between the a-posteriori LLRs and clean versions of the codewords with each batch; and
means for accessing encoded data from the non-volatile storage cells;
wherein the means for performing message passing computations on a batch of noisy versions of codewords is further for decoding the encoded data by performing the message passing computations with the revised one or more parameters over one or more iterations of the sparse bipartite graph.
19. The non-volatile storage device of claim 18 , wherein the means for revising the one or more parameters comprises means for performing the message passing computations on additional batches of noisy versions of codewords to attempt to minimize a bit error rate.