Rebuilding data via locally decodable redundancy in a vast storage network
In various embodiments, a method is presented for execution by a processing circuit of a storage network device that includes a processor. The method comprises: receiving, via the processing circuit, a first subset of encoded data slices of a set of encoded data slices corresponding to a data segment; identifying, via the processing circuit, at least one missing encoded data slice of the first subset of slices that is required to decode the data segment; generating, via the processing circuit, at least one rebuilt encoded data slice corresponding to the at least one missing encoded data slice utilizing locally decodable redundancy data; and recovering, via the processing circuit, the data segment based on the at least one rebuilt encoded data slice and the first subset of encoded data slices.
1. A method for execution by a processing circuit of a storage network device that includes a processor, the method comprises:
sending, to at least one storage unit of a storage network, read requests that identify encoded data slices of a set of encoded data slices to be retrieved, wherein the set of encoded data slices correspond to a data segment, wherein the data segment is decodable in accordance with dispersed error coding parameters that include a pillar width and a read threshold number, and wherein a number of encoded data slices in the set of encoded data slices corresponds to the pillar width;
receiving, via the processing circuit and from the at least one storage unit, a first subset of encoded data slices of the set of encoded data slices;
identifying, via the processing circuit, when the first subset of encoded data slices includes less than the read threshold number of encoded data slices of the set of encoded data and is missing at least one missing encoded data slice of the set of slices that is required to decode the data segment and that was not received from the at least one storage unit in response to the read requests;
generating, via the processing circuit, at least one rebuilt encoded data slice corresponding to the at least one missing encoded data slice utilizing locally decodable redundancy data, wherein the locally decodable redundancy data includes a plurality of local redundancy slices generated from a proper subset of the set of encoded data slices that includes the at least one missing encoded data slice; and
recovering, via the processing circuit, the data segment based on the at least one rebuilt encoded data slice and the first subset of encoded data slices.
2. The method of claim 1 , wherein the data segment is error encoded to produce the set of encoded data slices, and wherein a read threshold number of the set of encoded data slices are required to decode the data segment.
3. The method of claim 2 , wherein the processing circuit identifies the at least one missing encoded data slice of the first subset of encoded data slices when the first subset of encoded data slices includes less than the read threshold number of encoded data slices.
4. The method of claim 1 , further comprising:
selecting the proper subset of encoded data slices of the set of encoded data slices to generate the locally decodable redundancy data.
5. The method of claim 1 , further comprising:
locally storing the locally decodable redundancy data in a memory of the storage network device.
6. The method of claim 5 wherein the at least one storage unit is separate from the storage network device.
7. A processing system of a storage network device comprises:
a processing circuit;
a memory that stores operational instructions, that when executed by the processing circuit cause the processing system to perform operations that include:
sending, to at least one storage unit of a storage network, read requests that identify encoded data slices of a set of encoded data slices to be retrieved, wherein the set of encoded data slices correspond to a data segment, wherein the data segment is decodable in accordance with dispersed error coding parameters that include a pillar width and a read threshold number, and wherein a number of encoded data slices in the set of encoded data slices corresponds to the pillar width;
receiving, via the processing circuit and from the at least one storage unit, a first subset of encoded data slices of the set of encoded data slices;
identifying, via the processing circuit, when the first subset of encoded data slices includes less than the read threshold number of encoded data slices of the set of encoded data and is missing at least one missing encoded data slice of the set of slices that is required to decode the data segment and that was not received from the at least one storage unit in response to the read requests;
generating, via the processing circuit, at least one rebuilt encoded data slice corresponding to the at least one missing encoded data slice utilizing locally decodable redundancy data, wherein the locally decodable redundancy data includes a plurality of local redundancy slices generated from a proper subset of the set of encoded data slices that includes the at least one missing encoded data slice; and
recovering, via the processing circuit, the data segment based on the at least one rebuilt encoded data slice and the first subset of encoded data slices.
8. The processing system of claim 7 , wherein the data segment is error encoded to produce the set of encoded data slices, and wherein a read threshold number of the set of encoded data slices are required to decode the data segment.
9. The processing system of claim 8 , wherein the processing circuit identifies the at least one missing encoded data slice of the first subset of encoded data slices when the first subset of encoded data slices includes less than the read threshold number of encoded data slices.
10. The processing system of claim 7 , wherein the operations further include:
selecting the proper subset of encoded data slices of the set of encoded data slices to generate the locally decodable redundancy data.
11. The processing system of claim 7 , wherein the operations further include:
locally storing the locally decodable redundancy data in a memory of the storage network device.
12. The processing system of claim 11 , wherein the at least one storage unit is separate from the storage network device.
13. A non-transitory computer readable storage medium comprises:
at least one memory that stores operational instructions that, when executed by a processing system of a storage network device that includes a processing circuit and a memory, causes the processing system to perform operations that include:
sending, to at least one storage unit of a storage network, read requests that identify encoded data slices of a set of encoded data slices to be retrieved, wherein the set of encoded data slices correspond to a data segment, wherein the data segment is decodable in accordance with dispersed error coding parameters that include a pillar width and a read threshold number, and wherein a number of encoded data slices in the set of encoded data slices corresponds to the pillar width;
receiving, via the processing circuit and from the at least one storage unit, a first subset of encoded data slices of the set of encoded data slices;
identifying, via the processing circuit, when the first subset of encoded data slices includes less than the read threshold number of encoded data slices of the set of encoded data and is missing at least one missing encoded data slice of the set of slices that is required to decode the data segment and that was not received from the at least one storage unit in response to the read requests;
generating, via the processing circuit, at least one rebuilt encoded data slice corresponding to the at least one missing encoded data slice utilizing locally decodable redundancy data, wherein the locally decodable redundancy data includes a plurality of local redundancy slices generated from a proper subset of the set of encoded data slices that includes the at least one missing encoded data slice; and
recovering, via the processing circuit, the data segment based on the at least one rebuilt encoded data slice and the first subset of encoded data slices.
14. The non-transitory computer readable storage medium of claim 13 , wherein the data segment is error encoded to produce the set of encoded data slices, and wherein a read threshold number of the set of encoded data slices are required to decode the data segment.
15. The non-transitory computer readable storage medium of claim 14 , wherein the processing system identifies the at least one missing encoded data slice of the first subset of encoded data slices when the first subset of encoded data slices includes less than the read threshold number of encoded data slices.
16. The non-transitory computer readable storage medium of claim 13 , wherein the operations further include:
selecting the proper subset of encoded data slices of the set of encoded data slices to generate the locally decodable redundancy data.
17. The non-transitory computer readable storage medium of claim 13 , wherein the operations further include:
locally storing the locally decodable redundancy data in a memory of the storage network device.