Concurrent set storage in distributed storage network
View Patent ↗For each original data segment, a distributed storage processing unit generates encoded slices designed to prevent the original data segment from being reconstructed using fewer than a threshold number of encoded slices. Multiple encoded slices are generated for each of two different data segments, and the slices associated with the first and second data segment are stored substantially concurrently in different storage sets employing different distributed storage units. Encoded slices for even and odd data segments can be stored in different storage sets, or longer sequences of data segments can be stored in alternating storage sets. Storage sets can also be determined by the vault generation of a particular data segment.
1. A method for use in a distributed storage processing unit, the method comprising:
generating a first plurality of data slices including different encoded versions of a first data portion;
generating a second plurality of data slices including different encoded versions of a second data portion; and
sending the first plurality of data slices and the second plurality of data slices to be stored concurrently in different storage sets, each of the different storage sets including a plurality of distributed storage units different from the distributed storage units included in others of the different storage sets;
at least a read threshold number of data slices from a first storage set required to reconstruct the first data portion; and
at least a read threshold number of data slices from a second storage set required to reconstruct the second data portion.
2. The method of claim 1 , wherein the first data portion and the second data portion are data segments, the method further comprising:
sending data slices generated from even data segments to be stored in the first storage set; and
sending data slices generated from odd data segments to be stored in the second storage set.
3. The method of claim 1 , wherein the first data portion and the second data portion are data segments, the method further comprising:
sending data slices generated from a first sequence of data segments to be stored in the first storage set; and
storing data slices generated from a second sequence of data segments to be stored in the second storage set.
4. The method claim 1 , wherein the first data portion and the second data portion are different vault generations of a same data object, the method further comprising:
sending data slices generated from different generations of each of a plurality of data objects to be stored in different storage sets.
5. The method of claim 4 , further comprising:
dividing each of the plurality of data objects into a plurality of sub-files;
labeling the plurality of sub-files with a same file name, but with different vault generations; and
generating the first plurality of data slices and the second plurality of data slices based, at least in part on the different vault generations.
6. The method of claim 1 , further comprising:
determining which of a plurality of predetermined parallel storage schemes to implement based on a system parameter.
7. The method of claim 6 , wherein the system parameter is selected from the group consisting of data type, priority level, security level, preferred performance level, system loading indicator, and system configuration.
8. A distributed storage processing unit comprising:
a processor to generate a first plurality of data slices including different encoded versions of a first data portion and a second plurality of data slices including different encoded versions of a second data portion;
an interface to send the first plurality of data slices and the second plurality of data slices to be stored concurrently in different storage sets, each of the different storage sets including a plurality of distributed storage units different from the distributed storage units included in others of the different storage sets;
at least a read threshold number of data slices from a first storage set required to reconstruct the first data portion; and
at least a read threshold number of data slices from a second storage set required to reconstruct the second data portion.
9. The distributed storage processing unit of claim 8 , wherein the first data portion and the second data portion are data segments, the distributed storage processing unit further comprising:
the interface further to send data slices generated from even data segments to be stored in the first storage set and to send data slices generated from odd data segments to be stored in the second storage set.
10. The distributed storage processing unit of claim 8 , wherein the first data portion and the second data portion are data segments, the distributed storage processing unit further comprising:
the interface further to send data slices generated from a first sequence of data segments to be stored in the first storage set; and
the interface further to send data slices generated from a second sequence of data segments to be stored in the second storage set.
11. The distributed storage processing unit claim 8 , wherein the first data portion and the second data portion are different vault generations of a same data object, the distributed storage processing unit further comprising:
the interface further to send data slices generated from different generations of each of a plurality of data objects to be stored in different storage sets.
12. The distributed storage processing unit of claim 11 , further comprising:
the processor further to:
divide each of the plurality of data objects into a plurality of sub-files;
label the plurality of sub-files with a same file name, but with different vault generations; and
generate the first plurality of data slices and the second plurality of data slices based, at least in part on the different vault generations.
13. The distributed storage processing unit of claim 8 , further comprising:
the processor further to determine which of a plurality of predetermined parallel storage schemes to implement based on a system parameter.
14. The distributed storage processing unit of claim 13 , wherein the system parameter is selected from the group consisting of data type, priority level, security level, preferred performance level, system loading indicator, and system configuration.