Data recovery in a geographically diverse storage system employing erasure coding technology and data convolution technology
Data convolution for geographically diverse storage is disclosed. Data and corresponding convolutions of data can employ erasure coding to improve robustness of access to information represented in the data. For a peer group of chunks employing a given erasure coding scheme, access to the information represented in the data can be via accessible chunks and/or recovery of a less-accessible chunk, e.g., via a deconvolution operation, via a decoding operation, via a mix of deconvolution and decoding operations. The mix of deconvolution and decoding operations can enable recovery of a less-accessible chunk that cannot be recovered by either a deconvolution or decoding operation alone. This can improve access to information represented in less-available data.
1. A system, comprising:
a processor; and
a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, comprising:
determining a peer group of chunks stored via a geographically distributed data storage system, wherein the chunks of the peer group of chunks employ a same erasure coding scheme;
selecting a recovery scheme from a group of recovery schemes comprising a ‘deconvolve first’ scheme and a ‘decode first’ scheme, wherein the selecting is based on a criterion selected from a group of criteria comprising a count criterion based on a predicted count of operations to complete a corresponding recovery, a time criterion based on a predicted time to complete the corresponding recovery, a consumption criterion based on a predicted consumption of system resources to complete the corresponding recovery, and a monetary value criterion corresponding to data to recovered in the corresponding recovery;
in response to selecting the ‘deconvolve first’ scheme,
recovering a first fragment of a first chunk of the peer group of chunks based in part on a second fragment of a second chunk of the peer group of chunks, and
recovering a third fragment of the first chunk based in part on a fourth fragment of the first chunk; and
in response to selecting the ‘decode first’ scheme,
recovering a fifth fragment of the first chunk of the peer group of chunks based in part on a sixth fragment of the first chunk, and
recovering a seventh fragment of the second chunk based in part on an eighth fragment of the first chunk.
2. The system of claim 1 , wherein the chunks of the peer group of chunks employ a 12+4 erasure coding scheme.
3. The system of claim 1 , wherein the chunks of the peer group of chunks employ a 10+2 erasure coding scheme.
4. The system of claim 1 , wherein the chunks of the peer group of chunks comprise at least a third chunk, and wherein the second chunk results from comprising the first chunk and the at least the third chunk.
5. The system of claim 4 , wherein the third chunk results from convolving a fourth chunk and at least a fifth chunk.
6. The system of claim 5 , wherein the fourth chunk employs the same erasure coding scheme as the first chunk employs.
7. The system of claim 5 , wherein the fourth chunk employs a different erasure coding scheme than the first chunk employs.
8. The system of claim 5 , wherein the at least the fifth chunk employs the same erasure coding scheme as the first chunk employs.
9. The system of claim 5 , wherein the at least the fifth chunk employs a different erasure coding scheme as the first chunk employs.
10. The system of claim 4 , wherein the convolving the first chunk and the at least the third chunk comprises a chunk-level exclusive-or operation that occurs before employing the same erasure coding scheme.
11. The system of claim 4 , wherein the convolving the first chunk and the at least the third chunk comprises a fragment-level exclusive-or operation that occurs after employing the same erasure coding scheme.
12. A method, comprising:
in response to determining, by a system comprising a processor, that a portion of a peer group of chunks stored in a geographically diverse data storage system has become less available than a previous state of availability, determining a type of a first iteration of a recovery process, wherein all chunks comprising the peer group of chunks employ a same erasure coding process, wherein the determining the type of the first iteration of the recovery process is based on a predicted criterion selected from a group of criteria comprising a count criterion based on a predicted count of operations to complete a corresponding recovery comprising the first iteration of the recovery process, a time criterion based on a predicted time to complete the corresponding recovery, a consumption criterion based on a predicted consumption of system resources to complete the corresponding recovery, and a predicted monetary value criterion corresponding to the less available data;
in response to the type of the first iteration being a ‘deconvolve first’ type,
recovering, by the system, a first fragment of a first chunk of the peer group of chunks based in part on a second fragment of a second chunk of the peer group of chunks, and
recovering a third fragment of the first chunk based in part on a fourth fragment of the first chunk; and
in response to the type of the first iteration being a ‘decode first’ type,
recovering, by the system, a fifth fragment of the first chunk based in part on a sixth fragment of the first chunk, and
recovering a seventh fragment of the second chunk based in part on an eighth fragment of the first chunk.
13. The method of claim 12 , wherein the determining the type of the first iteration of the recovery process comprises determining that the first chunk comprises more than m less-available fragments.
14. The method of claim 13 , wherein the erasure coding process comprises k+m fragments, and wherein the erasure coding process is selected from a group of erasure coding schema comprising a 12+4 erasure coding process and a 10+2 erasure coding process.
15. The method of claim 12 , wherein the determining the type of the first iteration of the recovery process comprises determining that the first fragment of the first chunk is a corresponding fragment with the second fragment of the second chunk and is a corresponding fragment with a ninth fragment of a third chunk of the peer group of chunks, wherein the first fragment is a less-available fragment, wherein the second fragment is an available fragment, and wherein the ninth fragment is an available fragment.
16. The method of claim 12 , wherein the operations further comprise:
in response to determining, by the system, that the portion of the peer group of chunks remains less available than the previous state of availability, determining another type of a second iteration of the recovery process;
in response to the second iteration type being a ‘deconvolve first’ type,
recovering, by the system, ninth fragment of a third chunk of the peer group of chunks based in part on a tenth fragment of a fourth chunk of the peer group of chunks, and
recovering a eleventh fragment of the third chunk based in part on a twelfth fragment of the third chunk; and
in response to the second iteration type being a ‘decode first’ type,
recovering, by the system, a thirteenth fragment of the third chunk based in part on a fourteenth fragment of the third chunk, and
recovering a fifteenth fragment of the fourth chunk based in part on an sixteenth fragment of the third chunk.
17. A non-transitory machine-readable storage medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, comprising:
for a peer group of chunks stored via a geographically distributed data storage system, wherein the peer group of chunks comprises a first chunk, a second chunk, and a third chunk, wherein all chunks comprising the peer group of chunks employ a same erasure coding scheme, determining a first recovery scheme for recovering data, wherein the first recovery scheme comprises:
initially recovering a first fragment of a first chunk of the peer group of chunks based in part on a second fragment of a second chunk of the peer group of chunks, and
subsequently recovering a third fragment of the first chunk based in part on a fourth fragment of the first chunk;
determining a second recovery scheme for recovering the data, wherein the second recovery scheme comprises:
initially recovering a fifth fragment of the first chunk of the peer group of chunks based in part on a sixth fragment of the first chunk, and
subsequently recovering a seventh fragment of the second chunk based in part on an eighth fragment of the first chunk;
ranking recovery schema according to a determined performance, wherein the determined performance is based on a criterion selected from a group of criteria comprising a count criterion based on a predicted count of operations to complete a corresponding recovery scheme, and wherein the recovery schema comprise the first recovery scheme and the second recovery scheme; and
employing a selected recovery scheme to recover a portion of the peer group of chunks, wherein the selected recovery scheme is selected based on the ranking of the selected recovery scheme among the ranked recovery schema.
18. The non-transitory machine-readable storage medium of claim 17 , wherein the same erasure coding scheme is selected from erasure coding schema comprising a 12+4 erasure coding scheme and a 10+2 erasure coding scheme.
19. The non-transitory machine-readable storage medium of claim 17 , wherein the group of criteria further comprises a time criterion based on a predicted time to complete the corresponding recovery scheme, a consumption criterion based on a predicted consumption of system resources to complete the corresponding recovery scheme, and a monetary criterion based on a monetary value of the data criteria.
20. The non-transitory machine-readable storage medium of claim 17 , wherein the second chunk is a convolution of the first chunk and the third chunk.