Multistep recovery employing erasure coding in a geographically diverse data storage system
Multistep recovery of chunk fragments of a peer group employing hierarchical erasure coding for geographically diverse data storage protection is disclosed. A peer group of chunks can employ zone-level erasure coding of chunks that can each employ chunk-level erasure coding. In a first iteration, fragment recovery can be performed across peer group chunks based on the zone-level erasure coding. Subsequently, the first iteration can perform recovery of other fragments within a chunk based on the chunk-level erasure coding. Where additional fragments are to be recovered, subsequent iterations can be performed. The disclosed multistep recovery can enable recovery of fragments that would typically have been considered unrecoverable via conventional techniques. Additionally, multistep recovery can enable recovery of fragments across a peer group of chunks that can be more computing resource efficient than recovery of chunks across the peer group of chunks.
1. A system, comprising:
a processor; and
a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, comprising:
determining, in a first iteration, a peer group of chunks stored via a geographically distributed data storage system, wherein the chunks of the peer group of chunks employ a same chunk-level erasure coding scheme, and wherein the peer group of chunks employ a zone-level erasure coding scheme;
in response to selecting a ‘zone-level decoding first’ scheme,
recovering a first fragment of a first chunk of the peer group of chunks by employing a second fragment of a second chunk of the peer group of chunks via a first zone-level decoding operation, wherein the second fragment corresponds to the first fragment, and
recovering a third fragment of the first chunk by employing a fourth fragment of the first chunk via a first chunk-level decoding operation; and
in response to selecting a ‘chunk-level decoding first’ scheme,
recovering a fifth fragment of the first chunk of the peer group of chunks by employing a sixth fragment of the first chunk via a second chunk-level decoding operation, and
recovering a seventh fragment of the second chunk by employing an eighth fragment of the first chunk via a second zone-level decoding operation, wherein the eighth fragment corresponds to the seventh fragment.
2. The system of claim 1 , wherein the chunk-level erasure coding scheme is a 12+4 chunk-level erasure coding scheme.
3. The system of claim 1 , wherein the chunk-level erasure coding scheme is a 10+2 chunk-level erasure coding scheme.
4. The system of claim 1 , wherein the zone-level erasure coding scheme is a 4+2 zone-level erasure coding scheme.
5. The system of claim 1 , wherein the zone-level erasure coding scheme is a different erasure coding scheme than the chunk-level erasure coding scheme.
6. The system of claim 1 , wherein the zone-level erasure coding scheme is a same erasure coding scheme as the chunk-level erasure coding scheme.
7. The system of claim 1 , wherein the operations further comprise performing a second iteration comprising recovering a ninth fragment of the first chunk of the peer group of chunks by employing a tenth fragment of the second chunk of the peer group of chunks via a third zone-level decoding operation, and wherein the tenth fragment corresponds to the ninth fragment.
8. The system of claim 1 , wherein the operations further comprise performing a second iteration comprising recovering an eleventh fragment of the first chunk by employing a twelfth fragment of the first chunk via a second chunk-level decoding operation.
9. The system of claim 1 , wherein the operations further comprise performing subsequent iterations comprising subsequent zone-level recovery of a subsequent fragment among the chunks of the peer group of chunks and subsequent chunk-level recovery of another subsequent fragment within a chunk of the peer group chunks.
10. The system of claim 9 , wherein the subsequent iterations continue until all data fragments of all chunks of the peer group of chunks are recovered.
11. The system of claim 9 , wherein the subsequent iterations continue until a last iteration of the subsequent iterations fails due to a count of available fragments among chunks of the peer group of chunks being insufficient to perform a last subsequent zone-level fragment recovery operation.
12. The system of claim 9 , wherein the subsequent iterations continue until a last iteration of the subsequent iterations fails due to a count of available fragments within a chunk being insufficient to perform a last subsequent chunk-level fragment recovery operation.
13. A method, comprising:
performing, by a system comprising a processor, a first iteration of operations comprising:
in response to determining that a portion of a peer group of chunks stored in a geographically diverse data storage system has become unavailable, determining a type of a first iteration of a recovery process, wherein all chunks comprising the peer group of chunks employ a same chunk-level erasure coding scheme, and wherein the peer group of chunks employs a zone-level erasure coding scheme;
in response to the type being a ‘zone-level decoding first’ type:
recovering, by the system via a first zone-level decoding operation, a first fragment of a first chunk of the peer group of chunks based at least in part on a second fragment of a second chunk of the peer group of chunks, wherein the second fragment corresponds to the first fragment, and
recovering, via a first chunk-level decoding operation, a third fragment of the first chunk based at least in part on a fourth fragment of the first chunk; and
in response to determining, by the system, that an indicator satisfies a condition related to continuing operations, performing a second iteration of the operations subsequent to the first iteration.
14. The method of claim 13 , wherein the first iteration of operations further comprises:
in response to the type being a ‘chunk-level decoding first’ type:
recovering a fifth fragment of the first chunk of the peer group of chunks by employing a sixth fragment of the first chunk via a second chunk-level decoding operation, and
recovering a seventh fragment of the second chunk by employing an eighth fragment of the first chunk via a second zone-level decoding operation.
15. The method of claim 13 , wherein the zone-level erasure coding scheme results in the peer group of chunks comprising k zone +m zone chunks, and wherein the zone-level erasure coding scheme is selected from a group of erasure coding schema comprising a 12+4 erasure coding scheme, a 10+2 erasure coding scheme, and a 4+2 erasure coding scheme.
16. The method of claim 13 , wherein the chunk-level erasure coding scheme results in chunks comprising k chunk +m chunk fragments, and wherein the chunk-level erasure coding scheme is selected from a group of erasure coding schema comprising a 12+4 erasure coding scheme, a 10+2 erasure coding scheme, and a 4+2 erasure coding scheme.
17. The method of claim 13 , wherein the determining that the indicator satisfies a condition related to continuing operations comprises:
determining that all data fragments of the peer group of chunks have been recovered,
determining that a subsequent iteration of the operations cannot complete because a count of available fragments is less than k zone , resulting in prevention of completing a zone-level fragment recovery operation, or
determining that a subsequent iteration of the operations cannot complete because the count of available fragments is less than k chunk , resulting in prevention of a chunk-level fragment recovery operation.
18. A non-transitory machine-readable medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, comprising:
an initial iteration of operations comprising:
determining a type of a first iteration of a recovery process to facilitate recovery of a portion of a peer group of chunks stored in a geographically diverse data storage system, wherein all chunks comprising the peer group of chunks employ a same chunk-level erasure coding scheme, and wherein the peer group of chunks employs a zone-level erasure coding scheme;
in response to the type being a ‘zone-level decoding first’ type:
recovering, via a first zone-level decoding operation, a first fragment of a first chunk of the peer group of chunks based at least in part on a second fragment of a second chunk of the peer group of chunks, wherein the second fragment corresponds to the first fragment, and
recovering, via a first chunk-level decoding operation, a third fragment of the first chunk based at least in part on a fourth fragment of the first chunk; and
performing a subsequent iteration of the operations after the initial iteration.
19. The non-transitory machine-readable medium of claim 18 , wherein the initial iteration of operations further comprises:
in response to the type being a ‘chunk-level decoding first’ type:
recovering a fifth fragment of the first chunk of the peer group of chunks by employing a sixth fragment of the first chunk via a second chunk-level decoding operation, and
recovering a seventh fragment of the second chunk by employing an eighth fragment of the first chunk via a second zone-level decoding operation.
20. The non-transitory machine-readable medium of claim 18 , wherein the zone-level erasure coding scheme results in the peer group of chunks comprising k zone +m zone chunk, and wherein the chunk-level erasure coding scheme results in chunks comprising k chunk +m chunk fragments.