Recovering error corrected data
A plurality of storage nodes within a single chassis is provided. The plurality of storage nodes is configured to communicate together as a storage cluster. The plurality of storage nodes has a non-volatile solid-state storage for user data storage. The plurality of storage nodes is configured to distribute the user data and metadata associated with the user data throughout the plurality of storage nodes, with erasure coding of the user data. The plurality of storage nodes is configured to recover from failure of two of the plurality of storage nodes by applying the erasure coding to the user data from a remainder of the plurality of storage nodes. The plurality of storage nodes is configured to detect an error and engage in an error recovery via one of a processor of one of the plurality of storage nodes, a processor of the non-volatile solid state storage, or the flash memory.
1. A method comprising:
detecting a read error in reading data from solid-state memory of a storage system;
retrying the reading the data from the solid-state memory; and
determining error corrected data as a result of analysis of data from the retrying the reading of the data, wherein the determining is performed with assistance of a mirrored remote procedure call cache responsive to determination of a loss of a remote procedure call cache.
2. The method of claim 1 , wherein the retrying the reading the data from the solid-state memory comprises:
retrying the reading the data with differing levels of reference voltage in or to the solid-state memory.
3. The method of claim 1 , wherein the retrying the reading the data from the solid-state memory comprises:
retrying the reading the data with differing levels of reference current in or to the solid-state memory.
4. The method of claim 1 , wherein the retrying the reading the data from the solid-state memory comprises:
retrying the reading the data with error correction internal to a die in the solid-state memory.
5. The method of claim 1 , wherein the retrying the reading the data from the solid-state memory comprises:
retrying the reading the data with differing levels at which a read is performed in the solid-state memory.
6. The method of claim 1 , wherein the analysis includes applying one or more probabilistic calculations, that determine, for each bit in the error corrected data, whether the error corrected data bit is more likely to be a “1” or a “0”.
7. The method of claim 1 , comprising:
rebuilding the error corrected data in one of a plurality of storage nodes of the storage system, the rebuilt data utilizing an erasure coding differing from an erasure coding applied to data within the one of the plurality of storage nodes.
8. The method of claim 1 wherein the mirrored remote procedure call cache is located on a storage node of the storage system that is different than a storage node having the remote procedure call cache.
9. A non-transitory machine-readable medium having executable instructions to cause one or more processing units to perform a method for error recovery in a storage system, comprising:
detecting a read error in reading data from solid-state memory of the storage system;
retrying the reading the data from the solid-state memory; and
determining error corrected data as a result of analysis of data from the retrying the reading of the data, wherein the determining is performed with assistance of a mirrored remote procedure call cache responsive to determination of a loss of a remote procedure call cache.
10. The non-transitory machine-readable medium of claim 9 , wherein the retrying the reading the data from the solid-state memory comprises:
retrying the reading the data with differing levels of reference voltage in or to the solid-state memory.
11. The non-transitory machine-readable medium of claim 9 , wherein the retrying the reading the data from the solid-state memory comprises:
retrying the reading the data with differing levels of reference current in or to the solid-state memory.
12. The non-transitory machine-readable medium of claim 9 , wherein the retrying the reading the data from the solid-state memory comprises:
retrying the reading the data with error correction internal to a die in the solid-state memory.
13. The non-transitory machine-readable medium of claim 9 , wherein the retrying the reading the data from the solid-state memory comprises:
retrying the reading the data with differing levels at which a read is performed in the solid-state memory.
14. The non-transitory machine-readable medium of claim 9 , wherein the analysis includes applying one or more probabilistic calculations, that determine, for each bit in the error corrected data, whether the error corrected data bit is more likely to be a “1” or a “0”.
15. The non-transitory machine-readable medium of claim 9 , comprising:
rebuilding the error corrected data in one of a plurality of storage nodes of the storage system, the rebuilt data utilizing an erasure coding differing from an erasure coding applied to data within the one of the plurality of storage nodes.
16. The non-transitory machine-readable medium of claim 9 , wherein the mirrored remote procedure call cache is located on a storage node of the storage system that is different than a storage node having the remote procedure call cache.
17. A method, comprising:
detecting a read error in reading data from solid-state memory of a storage system;
retrying the reading the data from the solid-state memory; and
determining error corrected data as a result of analysis of data from the retrying the reading of the data, wherein the analysis includes applying one or more calculations, that determine, for each bit in the error corrected data, whether the error corrected data bit is more likely to be a “1” or a “0”.
18. The method of claim 17 , wherein the determining is performed with assistance of a mirrored remote procedure call cache responsive to determination of a loss of a remote procedure call cache.
19. The method of claim 18 wherein the mirrored remote procedure call cache is located on a storage node of the storage system that is different than a storage node having the remote procedure call cache.