LARGE OBJECT PARALLEL WRITING
A method includes partitioning a data object into a plurality of data partitions. The method further includes dispersed storage error encoding a first data partition of the plurality of data partitions into a first plurality of sets of encoded data slices. The method further includes generating a first segment allocation table (SAT) regarding storage of the first plurality of sets of encoded data slices in a first set of storage units of the DSN. The method further includes dispersed storage error encoding the first SAT to produce a first set of SAT slices. The method further includes sending the first plurality of sets of encoded data slices and the first set of SAT slices to the first set of storage units. The method further includes updating a directory with information regarding the first SAT.
1 . A method comprises:
partitioning, by a computing device of a dispersed storage network (DSN), a data object into a plurality of data partitions;
dispersed storage error encoding, by the computing device, a first data partition of the plurality of data partitions into a first plurality of sets of encoded data slices;
generating, by the computing device, a first segment allocation table (SAT) regarding storage of the first plurality of sets of encoded data slices in a first set of storage units of the DSN;
dispersed storage error encoding, by the computing device, the first SAT to produce a first set of SAT slices;
sending, by the computing device, the first plurality of sets of encoded data slices and the first set of SAT slices to the first set of storage units; and
updating, by the computing device, a directory with information regarding the first SAT.
2 . The method of claim 1 further comprises:
determining, by the computing device, to partition the data object based on one of:
a data object attribute;
a partitioning scheme lookup; and
receiving the partitioning scheme.
3 . The method of claim 1 , wherein the first SAT comprises:
a start segment vault source name entry indicating a vault source name associated with storage of a first set of encoded data slices of the first plurality of sets of encoded data slices, wherein a first data segment of one or more data segments of the first partition is dispersed error encoded into the first set of encoded data slices;
a segment size entry indicating a number of bytes of each data segment of the one or more data segments of the first partition; and
a total length entry indicating a number of bytes of all of the data segments of the one or more data segments of the first partition.
4 . The method of claim 1 further comprises:
generating a third SAT regarding storage of the first sets of SAT slices being stored in the first set of storage units and regarding storage of another set of SAT slices being stored in another set of storage units, wherein another SAT is regarding another plurality of encoding data slices, and wherein the another SAT is dispersed storage error encoded to produce the another set of SAT slices.
5 . The method of claim 4 , wherein the third SAT comprises:
a first data entry region including:
a start segment vault source name entry indicating a vault source name associated with storage of a first set of encoded data slices of the first plurality of sets of encoded data slices, wherein a first data segment of one or more data segments of the first data partition is dispersed error encoded into the first set of encoded data slices;
a segment size entry indicating a number of bytes of each data segment of the one or more data segments of the first data partition; and
a total length entry indicating a number of bytes of all of the data segments of the one or more data segments of the first data partition; and
another data entry region including:
a start segment vault source name entry indicating a vault source name associated with storage of a first set of encoded data slices of the other plurality of sets of encoded data slices, wherein a first data segment of one or more data segments of another partition of the plurality of data partitions is dispersed error encoded into the first set of encoded data slices;
a segment size entry indicating a number of bytes of each data segment of the one or more data segments of the other partition; and
a total length entry indicating a number of bytes of all of the data segments of the one or more data segments of the other partition.
6 . The method of claim 4 further comprises:
dispersed storage error encoding, by the computing device, the third SAT to produce a third set of SAT slices; and
sending, by the computing device, the third set of SAT slices to the first set of storage units or the other set of storage units.
7 . A computing device of a dispersed storage network (DSN), the computing device comprises:
an interface;
memory; and
a processing module operably coupled to the memory and the interface, wherein the processing module is operable to:
partition a data object into a plurality of data partitions;
dispersed storage error encode a first data partition of the plurality of data partitions into a first plurality of sets of encoded data slices;
generate a first segment allocation table (SAT) regarding storage of the first plurality of sets of encoded data slices in a first set of storage units of the DSN;
dispersed storage error encode the first SAT to produce a first set of SAT slices;
send the first plurality of sets of encoded data slices and the first set of SAT slices to the first set of storage units; and
update a directory with information regarding the first SAT.
8 . The computing device of claim 7 , wherein the processing module is operable to:
determine to partition the data object based on one of:
a data object attribute;
a partitioning scheme lookup; and
receiving the partitioning scheme.
9 . The computing device of claim 7 , wherein the first SAT comprises:
a start segment vault source name entry indicating a vault source name associated with storage of a first set of encoded data slices of the first plurality of sets of encoded data slices, wherein a first data segment of one or more data segments of the first partition is dispersed error encoded into the first set of encoded data slices;
a segment size entry indicating a number of bytes of each data segment of the one or more data segments of the first partition; and
a total length entry indicating a number of bytes of all of the data segments of the one or more data segments of the first partition.
10 . The computing device of claim 7 , wherein the processing module is operable to:
generate a third SAT regarding storage of the first sets of SAT slices being stored in the first set of storage units and regarding storage of another set of SAT slices being stored in another set of storage units, wherein another SAT is regarding another plurality of encoding data slices, and wherein the another SAT is dispersed storage error encoded to produce the another set of SAT slices.
11 . The computing device of claim 10 , wherein the other SAT comprises:
a start segment vault source name entry indicating a vault source name associated with storage of a first set of encoded data slices of the other plurality of sets of encoded data slices, wherein a first data segment of one or more data segments of another partition of the plurality of data partitions is dispersed error encoded into the first set of encoded data slices;
a segment size entry indicating a number of bytes of each data segment of the one or more data segments of the other partition; and
a total length entry indicating a number of bytes of all of the data segments of the one or more data segments of the other partition.
12 . The computing device of claim 10 , wherein the third SAT comprises:
a first data entry region including:
a start segment vault source name entry indicating a vault source name associated with storage of a first set of encoded data slices of the plurality of sets of encoded data slices, wherein a first data segment of one or more data segments of the first data partition is dispersed error encoded into the first set of encoded data slices;
a segment size entry indicating a number of bytes of each data segment of the one or more data segments of the first data partition; and
a total length entry indicating a number of bytes of all of the data segments of the one or more data segments of the first data partition; and
a second data entry region including:
a start segment vault source name entry indicating a vault source name associated with storage of a first set of encoded data slices of the other plurality of sets of encoded data slices, wherein a first data segment of one or more data segments of the other partition is dispersed error encoded into the first set of encoded data slices;
a segment size entry indicating a number of bytes of each data segment of the one or more data segments of the other partition; and
a total length entry indicating a number of bytes of all of the data segments of the one or more data segments of the other partition.
13 . The computing device of claim 10 , wherein the processing module is operable to:
dispersed storage error encode the third SAT to produce a third set of SAT slices; and
send the third set of SAT slices to the first set of storage units or the other set of storage units.