Enhanced dispersed storage error encoding using multiple encoding layers

1. A method comprises:

dispersed storage error encoding, by a computing device of a dispersed storage network (DSN), a data segment of a data file to produce a set of encoded data slices;

determining, by the computing device, a storage & error encoding scheme for storing the set of encoded data slices in a set of storage units of the DSN;

sending, by the computing device, the set of encoded data slices to the set of storage units in accordance with the storage & error encoding scheme;

receiving, by a first storage unit of the set of storage units, one or more encoded data slices of the set of encoded data slices;

processing, by the first storage unit, the one or more encoded data slices in accordance with a first version of the storage & error encoding scheme to produce a first set of encoded data sub-slices, wherein the first version of the storage & error encoding scheme is based on a set of memory devices of the first storage unit; and

storing, by the first storage unit, the first set of encoded data sub-slices in the set of memory devices of the first storage unit, wherein each encoded data sub-slice of the first set of data sub-slices is stored in a separate memory device of the set of memory devices of the first storage unit.

2. The method of claim 1 further comprises:

storing, by the first storage unit, a first encoded data slice of the one or more encoded data slices, wherein a threshold number of encoded data sub-slices of the first set of encoded data sub-slices is decodable to recover the first encoded data slice.

3. The method of claim 2 further comprises:

receiving, by the first storage unit, a read request for the first encoded data slice from a requesting device;

determining, by the first storage unit, whether the stored version of the first encoded data slice is error free;

when the stored version of the first encoded data slice is not error free, rebuilding, by the first storage unit, the first encoded data slice from the threshold number of encoded data sub-slices; and

sending, by the first storage unit, the rebuilt first encoded data slice to the requesting device.

4. The method of claim 1 further comprises:

receiving, by the first storage unit, a read request for a first encoded data slice of the one or more encoded data slices from a requesting device;

rebuilding, by the first storage unit, the first encoded data slice from the threshold number of encoded data sub-slices; and

sending, by the first storage unit, the rebuilt first encoded data slice to the requesting device.

5. The method of claim 1 further comprises:

receiving, by a second storage unit of the set of storage units, a second one or more encoded data slices of the set of encoded data slices;

processing, by the second storage unit, the second one or more encoded data slices in accordance with a second version of the storage & error encoding scheme to produce a second set of encoded data sub-slices, wherein the second version of the storage & error encoding scheme is based on a set of memory devices of the second storage unit; and

storing, by the second storage unit, the second set of encoded data sub-slices in the set of memory devices of the second storage unit, wherein each encoded data sub-slice of the second set of data sub-slices is stored in a separate memory device of the set of memory devices of the second storage unit.

6. The method of claim 5 further comprises:

the first version of the storage & error encoding scheme including a first dispersed storage error encoding function, wherein a pillar number of the first dispersed storage error encoding function corresponds to a number of memory devices of the set of memory devices of the first storage unit; and

the second version of the storage & error encoding scheme including a second dispersed storage error encoding function, wherein a pillar number of the second dispersed storage error encoding function corresponds to a number of memory devices of the set of memory devices of the second storage unit.

7. The method of claim 1 , wherein the processing the one or more encoded data slices comprises:

dispersed storage error encoding, by the first storage unit, a first encoded data slice of the one or more encoded data slices using a first dispersed storage error encoding function of the first version of the storage & error encoding scheme to produce the first set of encoded data sub-slices.

8. The method of claim 1 further comprises:

receiving, by the first storage unit, first and second encoded data slices of the set of encoded data slices as the one or more encoded data slices;

generating, by the first storage unit, a data matrix from the first and second encoded data slices;

matrix multiplying, by the first storage unit, the data matrix with an encoding matrix to produce a coded matrix; and

storing, by the first storage unit, portions of the coded matrix in the set of memory devices as the first set of encoded data sub-slices.

9. The method of claim 8 further comprises:

receiving, by the first storage unit, a read request for at least one of the first and second encoded data slices from a requesting device of the DSN;

recovering, by the first storage unit, the data matrix from the coded matrix and a decoding matrix;

recovering, by the first storage unit, the least one of the first and second encoded data slices from the data matrix; and

sending, by the first storage unit, the recovered at least one of the first and second encoded data slices to the requesting device.

10. A computer readable storage device comprises:

a first storage section that stores operational instructions that, when executed by a computing device of a dispersed storage network (DSN), causes the computing device to:

dispersed storage error encode a data segment of a data file to produce a set of encoded data slices;

determine a storage & error encoding scheme for storing the set of encoded data slices in a set of storage units of the DSN; and

send the set of encoded data slices to the set of storage units in accordance with the storage & error encoding scheme;

a second storage section that stores operational instructions that, when executed by a first storage unit of the set of storage units, causes the first storage unit to:

receive one or more encoded data slices of the set of encoded data slices;

process the one or more encoded data slices in accordance with a first version of the storage & error encoding scheme to produce a first set of encoded data sub-slices, wherein the first version of the storage & error encoding scheme is based on a set of memory devices of the first storage unit; and

store the first set of encoded data sub-slices in the set of memory devices of the first storage unit, wherein each encoded data sub-slice of the first set of data sub-slices is stored in a separate memory device of the set of memory devices of the first storage unit.

11. The computer readable storage device of claim 10 , where the second storage section further stores operational instructions that, when executed by the first storage unit, causes the first storage unit to:

store a first encoded data slice of the one or more encoded data slices, wherein a threshold number of encoded data sub-slices of the first set of encoded data sub-slices is decodable to recover the first encoded data slice.

12. The computer readable storage device of claim 11 further comprises:

a third storage section further stores operational instructions that, when executed by the first storage unit, causes the first storage unit to:

receive a read request for the first encoded data slice from a requesting device;

determine whether the stored version of the first encoded data slice is error free;

when the stored version of the first encoded data slice is not error free, rebuild the first encoded data slice from the threshold number of encoded data sub-slices; and

send the rebuilt first encoded data slice to the requesting device.

13. The computer readable storage device of claim 10 further comprises:

a third storage section further stores operational instructions that, when executed by the first storage unit, causes the first storage unit to:

receive a read request for a first encoded data slice of the one or more encoded data slices from a requesting device;

rebuild the first encoded data slice from the threshold number of encoded data sub-slices; and

send the rebuilt first encoded data slice to the requesting device.

14. The computer readable storage device of claim 10 further comprises:

a third storage section further stores operational instructions that, when executed by a second storage unit of the set of storage units, causes the second storage unit to:

receive a second one or more encoded data slices of the set of encoded data slices;

process the second one or more encoded data slices in accordance with a second version of the storage & error encoding scheme to produce a second set of encoded data sub-slices, wherein the second version of the storage & error encoding scheme is based on a set of memory devices of the second storage unit; and

store the second set of encoded data sub-slices in the set of memory devices of the second storage unit, wherein each encoded data sub-slice of the second set of data sub-slices is stored in a separate memory device of the set of memory devices of the second storage unit.

15. The computer readable storage device of claim 14 further comprises:

the first version of the storage & error encoding scheme including a first dispersed storage error encoding function, wherein a pillar number of the first dispersed storage error encoding function corresponds to a number of memory devices of the set of memory devices of the first storage unit; and

the second version of the storage & error encoding scheme including a second dispersed storage error encoding function, wherein a pillar number of the second dispersed storage error encoding function corresponds to a number of memory devices of the set of memory devices of the second storage unit.

16. The computer readable storage device of claim 10 , where the second storage section further stores operational instructions that, when executed by the first storage unit, causes the first storage unit to process the one or more encoded data slices by:

dispersed storage error encoding a first encoded data slice of the one or more encoded data slices using a first dispersed storage error encoding function of the first version of the storage & error encoding scheme to produce the first set of encoded data sub-slices.

17. The computer readable storage device of claim 10 , where the second storage section further stores operational instructions that, when executed by the first storage unit, causes the first storage unit to:

receive first and second encoded data slices of the set of encoded data slices as the one or more encoded data slices;

generate a data matrix from the first and second encoded data slices;

matrix multiply the data matrix with an encoding matrix to produce a coded matrix; and

store portions of the coded matrix in the set of memory devices as the first set of encoded data sub-slices.

18. The computer readable storage device of claim 17 further comprises:

a third storage section further stores operational instructions that, when executed by a second storage unit of the set of storage units, causes the second storage unit to:

receive a read request for at least one of the first and second encoded data slices from a requesting device of the DSN;

recover the data matrix from the coded matrix and a decoding matrix;

recover the least one of the first and second encoded data slices from the data matrix; and

send the recovered at least one of the first and second encoded data slices to the requesting device.