Method and apparatus for distributed storage integrity processing

1. A storage integrity processing unit in a distributed storage network, comprises:

a network interface for interfacing with a plurality of storage units in the distributed storage network; and

at least one rebuild module operable to:

receive a plurality of identifications of data slices that require rebuilding;

store in a rebuild queue the plurality of identifications of data slices that require rebuilding;

select one of the plurality of identifications of data slices from the rebuild queue based on a prioritization algorithm, wherein the selected data slice is one of an X number of a plurality of data slices of an encoded data segment and wherein the encoded data segment is one of a plurality of data segments of a data object and wherein the prioritization algorithm is based on a priority indicator associated with the data object;

reconstruct the encoded data segment from at least a number T of the X number of the plurality of data slices of the encoded data segment to generate a reconstructed data segment; and

generate a rebuilt encoded data slice corresponding to the selected data slice from the reconstructed data segment.

2. The storage integrity processing unit of claim 1 , wherein the priority indicator includes at least one of: a priority indicator for the data object, a priority indicator of a user associated with the data object, a priority indicator for a data object type and a data slice age.

3. The storage integrity processing unit of claim 1 , further comprising:

a slice queue operable to store the rebuilt encoded data slice, wherein the slice queue stores the rebuilt encoded data slice as one of a plurality of stored rebuilt data slices; and

a slice agent operable to:

select one of the plurality of stored rebuilt data slices based on the prioritization algorithm; and

store the selected one of the plurality of stored rebuilt data slices in one of the plurality of storage units.

4. The storage integrity processing unit of claim 1 , wherein the rebuild module is operable to receive the plurality of identifications of data slices that require rebuilding from one or more of the plurality of storage units.

5. The storage integrity processing unit of claim 1 , wherein portions of the encoded data segment are interleaved into a number X of a plurality of encoded data slices for storage in the plurality of storage units and wherein the identified data slice is one of the number X of the plurality of encoded data slices of the encoded data segment; and

wherein the number T corresponds to a threshold number of the number X of the plurality of encoded data slices needed to reconstruct the encoded data segment based on an error encoding dispersal function, wherein the number T equals less than the number X.

6. The storage integrity processing unit of claim 5 , wherein the rebuild module being operable to reconstruct the encoded data segment comprises:

being operable to reconstruct the data segment from at least the number T of the plurality of encoded data slices to produce the reconstructed data segment by decoding at least the number T of the plurality of encoded data slices using a decoding matrix A in accordance with the error encoding dispersal function.

7. The storage integrity processing unit of claim 6 , wherein the rebuild module being operable to decode at least the number T of the plurality of encoded data slices using a decoding matrix A in accordance with the error encoding dispersal function comprises:

being operable to deslice at least the number T of the plurality of encoded data slices to generate a portion of an encoded data segment; and

being operable to multiply the portion of the encoded data segment with the decoding matrix A, wherein the decoding matrix A is an MxR matrix, wherein the decoding matrix A includes a number R of rows equal to the number R of plurality of encoded data slices and includes a number M of columns equal to the number T of the plurality of encoded data slices needed to reconstruct the data segment based on the error encoding dispersal function.

8. A storage unit in a distributed storage network, comprising:

a network interface for interfacing with a plurality of additional storage units in the distributed storage network; and

at least one module operable to:

scan an address range of encoded data slices;

determine a plurality of identifications of encoded data slices that require rebuilding;

store in a rebuild queue the plurality of identifications of encoded data slices that require rebuilding;

select one of the plurality of identifications of encoded data slices from the rebuild queue to rebuild based on a prioritization algorithm, wherein the selected encoded data slice is one of an X number of encoded data slices associated with a data segment and wherein the prioritization algorithm is based on a priority indicator associated with the data segment;

retrieve at least a number T of the X number of the encoded data slices associated with the data segment, wherein the number T corresponds to a number of the plurality of encoded data slices needed to reconstruct the data segment based on an error encoding dispersal function;

reconstruct the data segment from at least the number T of the plurality of encoded data slices to produce a reconstructed data segment and

generate a rebuilt encoded data slice corresponding to the selected encoded data slice from the reconstructed data segment.

9. The storage unit of claim 8 , wherein the reconstructed data segment is one of a plurality of data segments of a data object.

10. The storage unit of claim 9 , wherein the priority indicator associated with the data segment includes a priority indicator associated with the data object.

11. The storage unit of claim 10 , wherein the priority indicator includes at least one of:

a priority indicator for the data object, a priority indicator of a user associated with the data object, a priority indicator for a data object type and a data slice age.

12. The storage unit of claim 8 , wherein the prioritization algorithm is based on a first in first out (FIFO) order.

13. The storage unit of claim 8 , wherein the module being operable to scan the address range of encoded data slices comprises:

being operable to scan an address range of encoded data slices, wherein the address range includes addresses assigned to the storage unit and to the plurality of additional storage units.

14. The storage unit of claim 13 , wherein the module being operable to retrieve at least the number T of the X number of the encoded data slices associated with the data segment comprises:

being operable to retrieve at least one of the number T of the number X of the encoded data slices associated with the data segment from at least one of the plurality of additional storage units.

15. A storage integrity system in a storage network, comprising:

one or more processing modules operable to:

access a rebuild queue, wherein the rebuild queue includes a plurality of identifications of data slices that require rebuilding;

select one of the plurality of identifications of data slices that requires rebuilding from the rebuild queue based on a prioritization algorithm, wherein the selected data slice is one of an X number of a plurality of data slices of an encoded data segment;

reconstruct the encoded data segment from at least a number T of the X number of the plurality of data slices of the encoded data segment to generate a reconstructed data segment, wherein the reconstructed data segment is one of a plurality of data segments of a data object and wherein the prioritization algorithm is based on a priority indicator associated with the data object; and

generate a rebuilt encoded data slice corresponding to the selected data slice from the reconstructed data segment.

16. The storage integrity system of claim 15 , wherein the priority indicator includes at least one of: a priority indicator for the data object, a priority indicator of a user associated with the data object, a priority indicator for a data object type and a data slice age.

17. The storage integrity system of claim 15 , wherein the prioritization algorithm is based on a first in first out (FIFO) order.

18. The storage integrity system of claim 15 , wherein the one or more processing modules are further operable to:

scan an address range of encoded data slices;

determine the plurality of identifications of encoded data slices that require rebuilding; and

store in the rebuild queue the plurality of identifications of encoded data slices that require rebuilding.